Jul 24 03:16:15.191855 ip-10-0-139-134 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 24 03:16:15.191868 ip-10-0-139-134 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 24 03:16:15.191897 ip-10-0-139-134 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 24 03:16:15.192209 ip-10-0-139-134 systemd[1]: Failed to start Kubernetes Kubelet. Jul 24 03:16:25.316448 ip-10-0-139-134 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 24 03:16:25.316466 ip-10-0-139-134 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 516f191832ad4e1a906d5581db6f0cb1 -- Jul 24 03:18:22.611962 ip-10-0-139-134 systemd[1]: Starting Kubernetes Kubelet... Jul 24 03:18:23.204462 ip-10-0-139-134 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 03:18:23.204462 ip-10-0-139-134 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 24 03:18:23.204462 ip-10-0-139-134 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 03:18:23.204462 ip-10-0-139-134 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 24 03:18:23.204462 ip-10-0-139-134 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 03:18:23.205243 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.205152 2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 24 03:18:23.210439 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210424 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 03:18:23.210439 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210439 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 03:18:23.210501 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210443 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 03:18:23.210501 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210446 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 03:18:23.210501 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210449 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 03:18:23.210501 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210452 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 03:18:23.210501 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210454 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 03:18:23.210501 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210457 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 03:18:23.210501 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210460 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 03:18:23.210501 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210464 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 03:18:23.210501 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210468 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 03:18:23.210501 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210472 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 03:18:23.210501 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210475 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 03:18:23.210501 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210478 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 03:18:23.210501 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210481 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 03:18:23.210501 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210484 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 03:18:23.210501 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210487 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 03:18:23.210501 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210490 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 03:18:23.210501 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210492 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 03:18:23.210501 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210495 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 03:18:23.210501 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210498 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 03:18:23.210966 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210500 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 03:18:23.210966 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210503 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 03:18:23.210966 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210506 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 03:18:23.210966 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210509 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 03:18:23.210966 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210511 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 03:18:23.210966 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210514 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 03:18:23.210966 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210517 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 03:18:23.210966 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210520 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 03:18:23.210966 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210522 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 03:18:23.210966 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210525 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 03:18:23.210966 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210528 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 03:18:23.210966 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210530 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 03:18:23.210966 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210533 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 03:18:23.210966 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210536 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 03:18:23.210966 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210538 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 03:18:23.210966 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210541 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 03:18:23.210966 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210543 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 03:18:23.210966 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210546 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 03:18:23.210966 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210548 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 03:18:23.210966 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210551 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 03:18:23.211439 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210554 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 03:18:23.211439 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210558 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 03:18:23.211439 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210560 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 03:18:23.211439 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210563 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 03:18:23.211439 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210565 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 03:18:23.211439 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210568 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 03:18:23.211439 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210571 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 03:18:23.211439 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210574 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 03:18:23.211439 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210576 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 03:18:23.211439 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210579 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 03:18:23.211439 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210581 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 03:18:23.211439 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210584 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 03:18:23.211439 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210586 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 03:18:23.211439 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210589 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 03:18:23.211439 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210592 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 03:18:23.211439 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210595 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 03:18:23.211439 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210597 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 03:18:23.211439 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210600 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 03:18:23.211439 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210602 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 03:18:23.211439 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210605 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 03:18:23.211947 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210607 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 03:18:23.211947 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210610 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 03:18:23.211947 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210614 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 03:18:23.211947 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210618 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 03:18:23.211947 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210621 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 03:18:23.211947 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210624 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 03:18:23.211947 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210626 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 03:18:23.211947 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210629 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 03:18:23.211947 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210632 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 03:18:23.211947 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210635 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 03:18:23.211947 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210637 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 03:18:23.211947 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210640 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 03:18:23.211947 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210643 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 03:18:23.211947 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210646 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 03:18:23.211947 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210648 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 03:18:23.211947 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210651 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 03:18:23.211947 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210653 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 03:18:23.211947 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210656 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 03:18:23.211947 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210658 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 03:18:23.212390 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210674 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 03:18:23.212390 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210677 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 03:18:23.212390 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210680 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 03:18:23.212390 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210682 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 03:18:23.212390 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210686 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 03:18:23.212390 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210689 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 03:18:23.212390 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210693 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 03:18:23.212390 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210695 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 03:18:23.212390 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210698 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 24 03:18:23.212390 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210701 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 03:18:23.212390 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210704 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 03:18:23.212390 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210706 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 03:18:23.212390 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210709 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 03:18:23.212390 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210712 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 03:18:23.212390 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210714 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 03:18:23.212390 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210717 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 03:18:23.212390 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210720 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 03:18:23.212390 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210723 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 03:18:23.212390 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.210727 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 03:18:23.212390 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211421 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 03:18:23.212911 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211427 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 03:18:23.212911 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211430 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 03:18:23.212911 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211433 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 03:18:23.212911 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211436 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 03:18:23.212911 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211439 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 03:18:23.212911 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211442 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 03:18:23.212911 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211445 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 03:18:23.212911 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211448 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 03:18:23.212911 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211451 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 03:18:23.212911 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211454 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 03:18:23.212911 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211456 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 03:18:23.212911 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211459 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 03:18:23.212911 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211464 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 03:18:23.212911 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211468 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 03:18:23.212911 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211471 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 03:18:23.212911 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211474 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 03:18:23.212911 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211477 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 03:18:23.212911 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211480 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 03:18:23.212911 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211483 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 03:18:23.212911 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211486 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 03:18:23.213416 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211488 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 03:18:23.213416 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211491 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 03:18:23.213416 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211494 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 03:18:23.213416 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211496 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 03:18:23.213416 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211499 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 03:18:23.213416 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211501 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 03:18:23.213416 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211504 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 03:18:23.213416 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211506 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 03:18:23.213416 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211509 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 03:18:23.213416 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211511 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 03:18:23.213416 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211514 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 03:18:23.213416 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211516 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 03:18:23.213416 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211519 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 03:18:23.213416 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211521 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 03:18:23.213416 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211524 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 03:18:23.213416 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211526 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 03:18:23.213416 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211529 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 03:18:23.213416 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211532 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 03:18:23.213416 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211535 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 03:18:23.213416 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211537 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 03:18:23.213921 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211540 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 03:18:23.213921 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211544 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 03:18:23.213921 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211547 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 24 03:18:23.213921 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211550 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 03:18:23.213921 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211553 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 03:18:23.213921 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211556 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 03:18:23.213921 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211559 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 03:18:23.213921 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211562 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 03:18:23.213921 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211565 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 03:18:23.213921 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211568 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 03:18:23.213921 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211571 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 03:18:23.213921 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211574 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 03:18:23.213921 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211576 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 03:18:23.213921 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211579 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 03:18:23.213921 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211582 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 03:18:23.213921 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211584 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 03:18:23.213921 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211587 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 03:18:23.213921 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211589 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 03:18:23.213921 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211592 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 03:18:23.214365 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211594 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 03:18:23.214365 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211597 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 03:18:23.214365 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211600 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 03:18:23.214365 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211602 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 03:18:23.214365 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211605 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 03:18:23.214365 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211607 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 03:18:23.214365 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211610 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 03:18:23.214365 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211612 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 03:18:23.214365 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211615 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 03:18:23.214365 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211618 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 03:18:23.214365 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211621 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 03:18:23.214365 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211623 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 03:18:23.214365 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211626 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 03:18:23.214365 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211629 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 03:18:23.214365 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211632 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 03:18:23.214365 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211635 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 03:18:23.214365 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211638 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 03:18:23.214365 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211641 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 03:18:23.214365 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211644 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 03:18:23.214820 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211647 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 03:18:23.214820 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211649 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 03:18:23.214820 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211653 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 03:18:23.214820 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211655 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 03:18:23.214820 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211658 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 03:18:23.214820 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211679 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 03:18:23.214820 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211684 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 03:18:23.214820 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211688 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 03:18:23.214820 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211692 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 03:18:23.214820 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211696 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 03:18:23.214820 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211700 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 03:18:23.214820 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211703 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 03:18:23.214820 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211706 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 03:18:23.214820 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211709 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 03:18:23.214820 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211712 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 03:18:23.214820 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211714 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 03:18:23.214820 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211717 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 03:18:23.214820 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211719 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 03:18:23.214820 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211722 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 03:18:23.214820 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.211724 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 03:18:23.215288 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212110 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 03:18:23.215288 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212115 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 03:18:23.215288 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212119 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 03:18:23.215288 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212122 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 03:18:23.215288 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212125 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 03:18:23.215288 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212128 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 03:18:23.215288 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212131 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 03:18:23.215288 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212133 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 03:18:23.215288 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212136 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 03:18:23.215288 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212139 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 03:18:23.215288 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212141 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 03:18:23.215288 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212144 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 03:18:23.215288 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212147 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 03:18:23.215288 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212150 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 03:18:23.215288 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212153 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 24 03:18:23.215288 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212156 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 03:18:23.215288 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212159 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 03:18:23.215288 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212162 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 03:18:23.215288 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212164 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 03:18:23.215288 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212167 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 03:18:23.215773 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212169 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 03:18:23.215773 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212172 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 03:18:23.215773 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212174 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 03:18:23.215773 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212177 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 03:18:23.215773 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212179 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 03:18:23.215773 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212182 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 03:18:23.215773 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212184 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 03:18:23.215773 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212187 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 03:18:23.215773 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212189 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 03:18:23.215773 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212192 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 03:18:23.215773 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212194 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 03:18:23.215773 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212197 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 03:18:23.215773 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212200 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 03:18:23.215773 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212203 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 03:18:23.215773 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212205 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 03:18:23.215773 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212208 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 03:18:23.215773 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212211 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 03:18:23.215773 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212214 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 03:18:23.215773 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212216 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 03:18:23.215773 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212218 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 03:18:23.216253 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212221 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 03:18:23.216253 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212223 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 03:18:23.216253 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212226 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 03:18:23.216253 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212229 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 03:18:23.216253 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212232 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 03:18:23.216253 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212235 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 03:18:23.216253 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212237 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 03:18:23.216253 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212240 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 03:18:23.216253 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212243 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 03:18:23.216253 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212245 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 03:18:23.216253 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212248 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 03:18:23.216253 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212250 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 03:18:23.216253 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212254 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 03:18:23.216253 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212258 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 03:18:23.216253 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212261 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 03:18:23.216253 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212264 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 03:18:23.216253 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212266 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 03:18:23.216253 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212269 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 03:18:23.216253 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212272 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 03:18:23.216727 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212275 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 03:18:23.216727 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212278 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 03:18:23.216727 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212281 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 03:18:23.216727 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212284 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 03:18:23.216727 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212286 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 03:18:23.216727 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212289 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 03:18:23.216727 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212292 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 03:18:23.216727 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212295 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 03:18:23.216727 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212299 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 03:18:23.216727 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212302 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 03:18:23.216727 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212305 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 03:18:23.216727 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212308 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 03:18:23.216727 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212311 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 03:18:23.216727 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212313 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 03:18:23.216727 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212316 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 03:18:23.216727 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212319 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 03:18:23.216727 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212322 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 03:18:23.216727 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212324 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 03:18:23.216727 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212327 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 03:18:23.217184 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212330 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 03:18:23.217184 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212333 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 03:18:23.217184 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212336 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 03:18:23.217184 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212338 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 03:18:23.217184 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212341 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 03:18:23.217184 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212343 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 03:18:23.217184 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212346 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 03:18:23.217184 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212348 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 03:18:23.217184 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212351 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 03:18:23.217184 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212353 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 03:18:23.217184 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212356 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 03:18:23.217184 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212359 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 03:18:23.217184 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212362 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 03:18:23.217184 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212364 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 03:18:23.217184 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212367 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 03:18:23.217184 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212369 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 03:18:23.217184 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212372 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 03:18:23.217184 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212375 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 03:18:23.217184 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212377 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 03:18:23.217184 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212380 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 03:18:23.217648 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.212382 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 03:18:23.217648 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212450 2579 flags.go:64] FLAG: --address="0.0.0.0" Jul 24 03:18:23.217648 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212457 2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 24 03:18:23.217648 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212463 2579 flags.go:64] FLAG: --anonymous-auth="true" Jul 24 03:18:23.217648 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212467 2579 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 24 03:18:23.217648 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212471 2579 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 24 03:18:23.217648 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212475 2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 24 03:18:23.217648 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212479 2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 24 03:18:23.217648 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212483 2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 24 03:18:23.217648 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212487 2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 24 03:18:23.217648 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212490 2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 24 03:18:23.217648 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212493 2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 24 03:18:23.217648 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212497 2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 24 03:18:23.217648 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212500 2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 24 03:18:23.217648 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212503 2579 flags.go:64] FLAG: --cgroup-root="" Jul 24 03:18:23.217648 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212506 2579 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 24 03:18:23.217648 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212509 2579 flags.go:64] FLAG: --client-ca-file="" Jul 24 03:18:23.217648 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212512 2579 flags.go:64] FLAG: --cloud-provider="external" Jul 24 03:18:23.217648 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212516 2579 flags.go:64] FLAG: --cluster-dns="[]" Jul 24 03:18:23.217648 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212520 2579 flags.go:64] FLAG: --cluster-domain="" Jul 24 03:18:23.217648 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212523 2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 24 03:18:23.217648 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212526 2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 24 03:18:23.217648 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212529 2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 24 03:18:23.217648 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212533 2579 flags.go:64] FLAG: --container-log-max-files="5" Jul 24 03:18:23.218233 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212536 2579 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 24 03:18:23.218233 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212540 2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 24 03:18:23.218233 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212543 2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 24 03:18:23.218233 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212546 2579 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 24 03:18:23.218233 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212549 2579 flags.go:64] FLAG: --contention-profiling="false" Jul 24 03:18:23.218233 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212552 2579 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 24 03:18:23.218233 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212555 2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 24 03:18:23.218233 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212558 2579 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 24 03:18:23.218233 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212562 2579 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 24 03:18:23.218233 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212568 2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 24 03:18:23.218233 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212571 2579 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 24 03:18:23.218233 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212575 2579 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 24 03:18:23.218233 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212578 2579 flags.go:64] FLAG: --enable-load-reader="false" Jul 24 03:18:23.218233 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212581 2579 flags.go:64] FLAG: --enable-server="true" Jul 24 03:18:23.218233 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212584 2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 24 03:18:23.218233 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212588 2579 flags.go:64] FLAG: --event-burst="100" Jul 24 03:18:23.218233 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212592 2579 flags.go:64] FLAG: --event-qps="50" Jul 24 03:18:23.218233 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212595 2579 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 24 03:18:23.218233 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212598 2579 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 24 03:18:23.218233 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212601 2579 flags.go:64] FLAG: --eviction-hard="" Jul 24 03:18:23.218233 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212605 2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 24 03:18:23.218233 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212608 2579 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 24 03:18:23.218233 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212611 2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 24 03:18:23.218233 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212614 2579 flags.go:64] FLAG: --eviction-soft="" Jul 24 03:18:23.218233 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212617 2579 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 24 03:18:23.218878 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212620 2579 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 24 03:18:23.218878 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212623 2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 24 03:18:23.218878 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212626 2579 flags.go:64] FLAG: --experimental-mounter-path="" Jul 24 03:18:23.218878 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212628 2579 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 24 03:18:23.218878 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212632 2579 flags.go:64] FLAG: --fail-swap-on="true" Jul 24 03:18:23.218878 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212634 2579 flags.go:64] FLAG: --feature-gates="" Jul 24 03:18:23.218878 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212638 2579 flags.go:64] FLAG: --file-check-frequency="20s" Jul 24 03:18:23.218878 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212641 2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 24 03:18:23.218878 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212644 2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 24 03:18:23.218878 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212647 2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 24 03:18:23.218878 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212650 2579 flags.go:64] FLAG: --healthz-port="10248" Jul 24 03:18:23.218878 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212653 2579 flags.go:64] FLAG: --help="false" Jul 24 03:18:23.218878 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212656 2579 flags.go:64] FLAG: --hostname-override="ip-10-0-139-134.ec2.internal" Jul 24 03:18:23.218878 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212659 2579 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 24 03:18:23.218878 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212674 2579 flags.go:64] FLAG: --http-check-frequency="20s" Jul 24 03:18:23.218878 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212678 2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 24 03:18:23.218878 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212684 2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 24 03:18:23.218878 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212687 2579 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 24 03:18:23.218878 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212690 2579 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 24 03:18:23.218878 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212693 2579 flags.go:64] FLAG: --image-service-endpoint="" Jul 24 03:18:23.218878 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212696 2579 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 24 03:18:23.218878 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212698 2579 flags.go:64] FLAG: --kube-api-burst="100" Jul 24 03:18:23.218878 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212701 2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 24 03:18:23.219480 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212704 2579 flags.go:64] FLAG: --kube-api-qps="50" Jul 24 03:18:23.219480 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212707 2579 flags.go:64] FLAG: --kube-reserved="" Jul 24 03:18:23.219480 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212710 2579 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 24 03:18:23.219480 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212713 2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 24 03:18:23.219480 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212716 2579 flags.go:64] FLAG: --kubelet-cgroups="" Jul 24 03:18:23.219480 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212719 2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 24 03:18:23.219480 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212722 2579 flags.go:64] FLAG: --lock-file="" Jul 24 03:18:23.219480 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212725 2579 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 24 03:18:23.219480 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212727 2579 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 24 03:18:23.219480 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212734 2579 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 24 03:18:23.219480 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212740 2579 flags.go:64] FLAG: --log-json-split-stream="false" Jul 24 03:18:23.219480 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212743 2579 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 24 03:18:23.219480 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212746 2579 flags.go:64] FLAG: --log-text-split-stream="false" Jul 24 03:18:23.219480 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212749 2579 flags.go:64] FLAG: --logging-format="text" Jul 24 03:18:23.219480 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212751 2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 24 03:18:23.219480 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212755 2579 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 24 03:18:23.219480 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212758 2579 flags.go:64] FLAG: --manifest-url="" Jul 24 03:18:23.219480 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212761 2579 flags.go:64] FLAG: --manifest-url-header="" Jul 24 03:18:23.219480 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212766 2579 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 24 03:18:23.219480 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212769 2579 flags.go:64] FLAG: --max-open-files="1000000" Jul 24 03:18:23.219480 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212773 2579 flags.go:64] FLAG: --max-pods="110" Jul 24 03:18:23.219480 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212776 2579 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 24 03:18:23.219480 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212779 2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 24 03:18:23.219480 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212782 2579 flags.go:64] FLAG: --memory-manager-policy="None" Jul 24 03:18:23.219480 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212785 2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 24 03:18:23.220106 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212789 2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 24 03:18:23.220106 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212792 2579 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 24 03:18:23.220106 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212802 2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 24 03:18:23.220106 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212810 2579 flags.go:64] FLAG: --node-status-max-images="50" Jul 24 03:18:23.220106 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212813 2579 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 24 03:18:23.220106 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212816 2579 flags.go:64] FLAG: --oom-score-adj="-999" Jul 24 03:18:23.220106 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212819 2579 flags.go:64] FLAG: --pod-cidr="" Jul 24 03:18:23.220106 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212822 2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 24 03:18:23.220106 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212827 2579 flags.go:64] FLAG: --pod-manifest-path="" Jul 24 03:18:23.220106 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212830 2579 flags.go:64] FLAG: --pod-max-pids="-1" Jul 24 03:18:23.220106 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212833 2579 flags.go:64] FLAG: --pods-per-core="0" Jul 24 03:18:23.220106 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212836 2579 flags.go:64] FLAG: --port="10250" Jul 24 03:18:23.220106 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212839 2579 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 24 03:18:23.220106 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212842 2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f5a54c589128cf46" Jul 24 03:18:23.220106 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212845 2579 flags.go:64] FLAG: --qos-reserved="" Jul 24 03:18:23.220106 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212848 2579 flags.go:64] FLAG: --read-only-port="10255" Jul 24 03:18:23.220106 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212851 2579 flags.go:64] FLAG: --register-node="true" Jul 24 03:18:23.220106 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212855 2579 flags.go:64] FLAG: --register-with-taints="" Jul 24 03:18:23.220106 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212859 2579 flags.go:64] FLAG: --registry-burst="10" Jul 24 03:18:23.220106 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212862 2579 flags.go:64] FLAG: --registry-qps="5" Jul 24 03:18:23.220106 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212864 2579 flags.go:64] FLAG: --reserved-cpus="" Jul 24 03:18:23.220106 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212867 2579 flags.go:64] FLAG: --reserved-memory="" Jul 24 03:18:23.220106 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212871 2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 24 03:18:23.220106 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212875 2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 24 03:18:23.220106 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212878 2579 flags.go:64] FLAG: --rotate-certificates="false" Jul 24 03:18:23.221074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212881 2579 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 24 03:18:23.221074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212884 2579 flags.go:64] FLAG: --runonce="false" Jul 24 03:18:23.221074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212887 2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 24 03:18:23.221074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212890 2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 24 03:18:23.221074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212893 2579 flags.go:64] FLAG: --seccomp-default="false" Jul 24 03:18:23.221074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212896 2579 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 24 03:18:23.221074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212899 2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 24 03:18:23.221074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212902 2579 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 24 03:18:23.221074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212906 2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 24 03:18:23.221074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212909 2579 flags.go:64] FLAG: --storage-driver-password="root" Jul 24 03:18:23.221074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212913 2579 flags.go:64] FLAG: --storage-driver-secure="false" Jul 24 03:18:23.221074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212916 2579 flags.go:64] FLAG: --storage-driver-table="stats" Jul 24 03:18:23.221074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212919 2579 flags.go:64] FLAG: --storage-driver-user="root" Jul 24 03:18:23.221074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212922 2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 24 03:18:23.221074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212925 2579 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 24 03:18:23.221074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212928 2579 flags.go:64] FLAG: --system-cgroups="" Jul 24 03:18:23.221074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212931 2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 24 03:18:23.221074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212936 2579 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 24 03:18:23.221074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212939 2579 flags.go:64] FLAG: --tls-cert-file="" Jul 24 03:18:23.221074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212941 2579 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 24 03:18:23.221074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212946 2579 flags.go:64] FLAG: --tls-min-version="" Jul 24 03:18:23.221074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212948 2579 flags.go:64] FLAG: --tls-private-key-file="" Jul 24 03:18:23.221074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212951 2579 flags.go:64] FLAG: --topology-manager-policy="none" Jul 24 03:18:23.221074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212954 2579 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 24 03:18:23.221074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212957 2579 flags.go:64] FLAG: --topology-manager-scope="container" Jul 24 03:18:23.221793 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212961 2579 flags.go:64] FLAG: --v="2" Jul 24 03:18:23.221793 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212964 2579 flags.go:64] FLAG: --version="false" Jul 24 03:18:23.221793 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212968 2579 flags.go:64] FLAG: --vmodule="" Jul 24 03:18:23.221793 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212972 2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 24 03:18:23.221793 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.212975 2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 24 03:18:23.221793 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213091 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 03:18:23.221793 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213095 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 03:18:23.221793 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213100 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 03:18:23.221793 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213103 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 03:18:23.221793 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213106 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 03:18:23.221793 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213109 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 03:18:23.221793 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213112 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 03:18:23.221793 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213115 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 03:18:23.221793 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213118 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 03:18:23.221793 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213121 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 03:18:23.221793 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213125 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 03:18:23.221793 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213128 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 03:18:23.221793 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213131 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 03:18:23.221793 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213134 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 03:18:23.221793 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213136 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 03:18:23.221793 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213139 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 03:18:23.222357 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213142 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 03:18:23.222357 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213144 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 03:18:23.222357 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213147 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 03:18:23.222357 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213150 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 03:18:23.222357 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213152 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 03:18:23.222357 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213155 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 03:18:23.222357 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213158 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 03:18:23.222357 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213160 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 03:18:23.222357 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213163 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 03:18:23.222357 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213165 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 03:18:23.222357 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213168 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 03:18:23.222357 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213171 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 03:18:23.222357 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213174 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 03:18:23.222357 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213176 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 03:18:23.222357 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213179 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 03:18:23.222357 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213181 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 03:18:23.222357 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213185 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 03:18:23.222357 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213188 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 03:18:23.222357 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213191 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 03:18:23.222357 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213193 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 03:18:23.222897 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213196 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 03:18:23.222897 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213198 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 03:18:23.222897 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213201 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 03:18:23.222897 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213203 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 03:18:23.222897 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213206 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 03:18:23.222897 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213208 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 24 03:18:23.222897 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213212 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 03:18:23.222897 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213215 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 03:18:23.222897 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213217 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 03:18:23.222897 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213220 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 03:18:23.222897 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213223 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 03:18:23.222897 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213225 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 03:18:23.222897 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213228 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 03:18:23.222897 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213231 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 03:18:23.222897 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213233 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 03:18:23.222897 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213236 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 03:18:23.222897 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213238 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 03:18:23.222897 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213241 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 03:18:23.222897 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213243 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 03:18:23.222897 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213246 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 03:18:23.223376 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213249 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 03:18:23.223376 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213251 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 03:18:23.223376 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213254 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 03:18:23.223376 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213256 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 03:18:23.223376 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213259 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 03:18:23.223376 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213262 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 03:18:23.223376 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213265 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 03:18:23.223376 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213267 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 03:18:23.223376 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213271 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 03:18:23.223376 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213273 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 03:18:23.223376 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213276 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 03:18:23.223376 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213278 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 03:18:23.223376 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213281 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 03:18:23.223376 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213283 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 03:18:23.223376 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213286 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 03:18:23.223376 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213289 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 03:18:23.223376 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213291 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 03:18:23.223376 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213294 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 03:18:23.223376 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213298 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 03:18:23.223376 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213301 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 03:18:23.223876 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213303 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 03:18:23.223876 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213306 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 03:18:23.223876 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213308 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 03:18:23.223876 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213311 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 03:18:23.223876 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213313 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 03:18:23.223876 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213316 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 03:18:23.223876 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213319 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 03:18:23.223876 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213322 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 03:18:23.223876 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213324 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 03:18:23.223876 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213327 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 03:18:23.223876 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213329 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 03:18:23.223876 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213332 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 03:18:23.223876 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213334 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 03:18:23.223876 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213339 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 03:18:23.223876 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213342 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 03:18:23.223876 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213345 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 03:18:23.223876 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213348 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 03:18:23.223876 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213351 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 03:18:23.223876 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213355 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 03:18:23.224333 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213358 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 03:18:23.224333 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213365 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 03:18:23.224333 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213368 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 03:18:23.224333 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.213371 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 03:18:23.224333 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.213985 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 03:18:23.224333 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.222118 2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 24 03:18:23.224333 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.222133 2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 24 03:18:23.224333 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.222152 2579 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 24 03:18:23.224333 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.222158 2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 24 03:18:23.224333 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222213 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 03:18:23.224333 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222219 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 03:18:23.224333 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222222 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 03:18:23.224333 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222226 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 03:18:23.224333 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222229 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 03:18:23.224333 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222231 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 03:18:23.224721 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222235 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 03:18:23.224721 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222237 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 03:18:23.224721 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222240 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 03:18:23.224721 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222243 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 03:18:23.224721 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222245 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 03:18:23.224721 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222248 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 03:18:23.224721 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222250 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 03:18:23.224721 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222253 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 03:18:23.224721 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222255 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 03:18:23.224721 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222258 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 03:18:23.224721 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222260 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 03:18:23.224721 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222263 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 03:18:23.224721 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222267 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 03:18:23.224721 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222271 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 03:18:23.224721 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222275 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 03:18:23.224721 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222277 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 03:18:23.224721 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222280 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 03:18:23.224721 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222283 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 03:18:23.224721 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222286 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 03:18:23.225191 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222288 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 03:18:23.225191 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222291 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 03:18:23.225191 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222294 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 03:18:23.225191 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222297 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 03:18:23.225191 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222300 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 03:18:23.225191 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222304 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 03:18:23.225191 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222307 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 03:18:23.225191 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222309 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 03:18:23.225191 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222312 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 03:18:23.225191 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222316 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 03:18:23.225191 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222320 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 03:18:23.225191 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222323 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 03:18:23.225191 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222326 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 03:18:23.225191 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222329 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 03:18:23.225191 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222332 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 03:18:23.225191 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222334 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 03:18:23.225191 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222337 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 03:18:23.225191 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222339 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 03:18:23.225191 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222342 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 03:18:23.225785 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222345 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 03:18:23.225785 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222347 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 03:18:23.225785 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222351 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 03:18:23.225785 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222353 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 03:18:23.225785 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222356 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 03:18:23.225785 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222358 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 03:18:23.225785 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222362 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 03:18:23.225785 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222364 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 03:18:23.225785 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222367 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 03:18:23.225785 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222370 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 03:18:23.225785 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222373 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 03:18:23.225785 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222376 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 03:18:23.225785 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222378 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 03:18:23.225785 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222380 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 03:18:23.225785 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222384 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 03:18:23.225785 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222387 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 03:18:23.225785 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222389 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 03:18:23.225785 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222392 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 03:18:23.225785 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222395 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 03:18:23.225785 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222398 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 03:18:23.226275 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222400 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 03:18:23.226275 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222403 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 03:18:23.226275 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222405 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 03:18:23.226275 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222408 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 03:18:23.226275 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222410 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 03:18:23.226275 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222413 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 03:18:23.226275 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222415 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 03:18:23.226275 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222418 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 03:18:23.226275 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222421 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 03:18:23.226275 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222423 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 03:18:23.226275 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222426 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 03:18:23.226275 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222430 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 03:18:23.226275 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222433 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 03:18:23.226275 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222436 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 03:18:23.226275 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222439 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 03:18:23.226275 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222442 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 03:18:23.226275 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222445 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 03:18:23.226275 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222448 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 03:18:23.226275 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222451 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 03:18:23.226798 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222453 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 03:18:23.226798 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222456 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 03:18:23.226798 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222459 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 03:18:23.226798 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222462 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 03:18:23.226798 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222464 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 03:18:23.226798 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222467 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 03:18:23.226798 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222469 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 03:18:23.226798 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222472 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 03:18:23.226798 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222475 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 03:18:23.226798 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222478 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 03:18:23.226798 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222481 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 03:18:23.226798 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222483 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 03:18:23.226798 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222486 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 03:18:23.226798 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222488 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 03:18:23.226798 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222491 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 24 03:18:23.226798 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222494 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 03:18:23.227200 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.222499 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 03:18:23.227200 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222609 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 03:18:23.227200 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222614 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 03:18:23.227200 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222617 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 03:18:23.227200 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222620 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 03:18:23.227200 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222623 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 03:18:23.227200 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222626 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 03:18:23.227200 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222629 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 03:18:23.227200 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222632 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 03:18:23.227200 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222635 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 03:18:23.227200 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222638 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 03:18:23.227200 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222640 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 03:18:23.227200 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222643 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 03:18:23.227200 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222647 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 03:18:23.227200 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222649 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 03:18:23.227569 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222653 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 03:18:23.227569 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222655 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 03:18:23.227569 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222658 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 03:18:23.227569 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222677 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 03:18:23.227569 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222681 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 03:18:23.227569 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222684 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 03:18:23.227569 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222687 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 03:18:23.227569 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222689 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 03:18:23.227569 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222692 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 03:18:23.227569 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222695 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 03:18:23.227569 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222698 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 03:18:23.227569 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222702 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 03:18:23.227569 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222707 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 03:18:23.227569 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222711 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 03:18:23.227569 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222714 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 03:18:23.227569 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222717 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 03:18:23.227569 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222721 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 03:18:23.227569 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222724 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 03:18:23.228014 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222727 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 03:18:23.228014 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222730 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 03:18:23.228014 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222733 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 03:18:23.228014 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222736 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 03:18:23.228014 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222738 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 03:18:23.228014 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222741 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 03:18:23.228014 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222743 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 03:18:23.228014 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222746 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 03:18:23.228014 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222749 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 03:18:23.228014 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222751 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 03:18:23.228014 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222754 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 03:18:23.228014 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222756 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 03:18:23.228014 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222759 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 03:18:23.228014 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222762 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 03:18:23.228014 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222765 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 03:18:23.228014 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222768 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 03:18:23.228014 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222770 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 03:18:23.228014 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222773 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 03:18:23.228014 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222776 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 24 03:18:23.228014 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222778 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 03:18:23.228490 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222781 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 03:18:23.228490 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222783 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 03:18:23.228490 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222786 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 03:18:23.228490 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222789 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 03:18:23.228490 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222791 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 03:18:23.228490 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222794 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 03:18:23.228490 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222797 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 03:18:23.228490 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222799 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 03:18:23.228490 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222802 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 03:18:23.228490 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222804 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 03:18:23.228490 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222807 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 03:18:23.228490 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222810 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 03:18:23.228490 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222812 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 03:18:23.228490 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222815 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 03:18:23.228490 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222817 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 03:18:23.228490 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222820 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 03:18:23.228490 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222822 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 03:18:23.228490 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222825 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 03:18:23.228490 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222827 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 03:18:23.228490 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222830 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 03:18:23.228990 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222832 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 03:18:23.228990 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222835 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 03:18:23.228990 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222837 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 03:18:23.228990 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222840 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 03:18:23.228990 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222842 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 03:18:23.228990 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222845 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 03:18:23.228990 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222849 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 03:18:23.228990 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222851 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 03:18:23.228990 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222854 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 03:18:23.228990 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222856 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 03:18:23.228990 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222859 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 03:18:23.228990 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222862 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 03:18:23.228990 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222864 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 03:18:23.228990 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222867 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 03:18:23.228990 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222869 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 03:18:23.228990 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222872 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 03:18:23.228990 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222874 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 03:18:23.228990 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222877 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 03:18:23.228990 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222880 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 03:18:23.228990 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222883 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 03:18:23.229487 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222885 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 03:18:23.229487 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222888 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 03:18:23.229487 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222890 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 03:18:23.229487 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222893 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 03:18:23.229487 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222895 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 03:18:23.229487 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222898 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 03:18:23.229487 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:23.222901 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 03:18:23.229487 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.222906 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 03:18:23.229487 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.223001 2579 server.go:962] "Client rotation is on, will bootstrap in background" Jul 24 03:18:23.229487 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.225325 2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 24 03:18:23.229487 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.226687 2579 server.go:1019] "Starting client certificate rotation" Jul 24 03:18:23.229487 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.226793 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 24 03:18:23.229487 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.226835 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 24 03:18:23.229893 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.229127 2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 24 03:18:23.230282 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.230265 2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 24 03:18:23.241853 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.241836 2579 log.go:25] "Validated CRI v1 runtime API" Jul 24 03:18:23.243911 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.243897 2579 log.go:25] "Validated CRI v1 image API" Jul 24 03:18:23.244635 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.244622 2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 24 03:18:23.247187 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.247169 2579 fs.go:135] Filesystem UUIDs: map[3c5d394a-d84d-4b8a-ae9e-96650c01ad3e:/dev/nvme0n1p3 3cffef29-9a62-49ea-93a0-a8b8a9e341d1:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 24 03:18:23.247235 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.247185 2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 24 03:18:23.252987 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.252872 2579 manager.go:216] Machine: {Timestamp:2026-07-24 03:18:23.250617155 +0000 UTC m=+0.526188497 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099601 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2138a9d586ca949692bbf40d40134b SystemUUID:ec2138a9-d586-ca94-9692-bbf40d40134b BootID:516f1918-32ad-4e1a-906d-5581db6f0cb1 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:17:5c:ef:08:19 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:17:5c:ef:08:19 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:16:ac:b3:26:b1:af Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 24 03:18:23.252987 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.252986 2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 24 03:18:23.253102 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.253067 2579 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 24 03:18:23.255832 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.255748 2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 24 03:18:23.255929 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.255843 2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 24 03:18:23.256254 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.255834 2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-134.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 24 03:18:23.257658 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.257637 2579 topology_manager.go:138] "Creating topology manager with none policy" Jul 24 03:18:23.257658 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.257676 2579 container_manager_linux.go:309] "Creating device plugin manager" Jul 24 03:18:23.257798 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.257689 2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 24 03:18:23.258474 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.258439 2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 24 03:18:23.259726 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.259714 2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 24 03:18:23.261427 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.261415 2579 state_mem.go:36] "Initialized new in-memory state store" Jul 24 03:18:23.261573 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.261564 2579 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 24 03:18:23.261603 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.261598 2579 kubelet.go:486] "Attempting to sync node with API server" Jul 24 03:18:23.261629 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.261610 2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 24 03:18:23.261629 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.261625 2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 24 03:18:23.261698 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.261633 2579 kubelet.go:398] "Adding apiserver pod source" Jul 24 03:18:23.261698 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.261642 2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 24 03:18:23.262031 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.262018 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 24 03:18:23.262066 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.262042 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 24 03:18:23.266086 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.266069 2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 24 03:18:23.268186 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.268171 2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 24 03:18:23.268263 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.268208 2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 24 03:18:23.269801 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.269769 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 24 03:18:23.269801 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.269793 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 24 03:18:23.269801 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.269802 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 24 03:18:23.269957 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.269812 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 24 03:18:23.269957 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.269827 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 24 03:18:23.269957 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.269836 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 24 03:18:23.269957 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.269849 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 24 03:18:23.269957 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.269857 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 24 03:18:23.269957 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.269868 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 24 03:18:23.269957 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.269876 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 24 03:18:23.269957 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.269888 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 24 03:18:23.269957 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.269896 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 24 03:18:23.269957 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:23.269890 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 24 03:18:23.269957 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:23.269927 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-134.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 24 03:18:23.270606 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.270595 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 24 03:18:23.270606 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.270607 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 24 03:18:23.272315 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.272297 2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-134.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 24 03:18:23.273630 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.273616 2579 server.go:1268] "Started kubelet" Jul 24 03:18:23.273742 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.273720 2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 24 03:18:23.273986 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.273940 2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 24 03:18:23.274054 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.273998 2579 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 24 03:18:23.274497 ip-10-0-139-134 systemd[1]: Started Kubernetes Kubelet. Jul 24 03:18:23.275891 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.275876 2579 server.go:310] "Adding debug handlers to kubelet server" Jul 24 03:18:23.276006 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.275991 2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 24 03:18:23.280198 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:23.280176 2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 24 03:18:23.280835 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.280816 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mvwfh" Jul 24 03:18:23.285244 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.285227 2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 24 03:18:23.285244 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.285235 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 24 03:18:23.285414 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:23.285390 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-134.ec2.internal\" not found" Jul 24 03:18:23.285979 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.285926 2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 24 03:18:23.285979 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.285928 2579 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 24 03:18:23.285979 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.285951 2579 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 24 03:18:23.286143 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.286121 2579 reconstruct.go:97] "Volume reconstruction finished" Jul 24 03:18:23.286143 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.286130 2579 reconciler.go:29] "Reconciler: start to sync state" Jul 24 03:18:23.286228 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.286158 2579 factory.go:55] Registering systemd factory Jul 24 03:18:23.286228 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.286172 2579 factory.go:223] Registration of the systemd container factory successfully Jul 24 03:18:23.286444 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.286429 2579 factory.go:153] Registering CRI-O factory Jul 24 03:18:23.286486 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.286447 2579 factory.go:223] Registration of the crio container factory successfully Jul 24 03:18:23.286530 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.286496 2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 24 03:18:23.286530 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.286528 2579 factory.go:103] Registering Raw factory Jul 24 03:18:23.286596 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.286539 2579 manager.go:1196] Started watching for new ooms in manager Jul 24 03:18:23.287194 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:23.287176 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 24 03:18:23.287289 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.287275 2579 manager.go:349] Starting recovery of all containers Jul 24 03:18:23.287381 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:23.287354 2579 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-134.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 24 03:18:23.288019 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:23.286960 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-134.ec2.internal.18c51b892b52d52c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-134.ec2.internal,UID:ip-10-0-139-134.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-134.ec2.internal,},FirstTimestamp:2026-07-24 03:18:23.273588012 +0000 UTC m=+0.549159354,LastTimestamp:2026-07-24 03:18:23.273588012 +0000 UTC m=+0.549159354,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-134.ec2.internal,}" Jul 24 03:18:23.288833 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.288810 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mvwfh" Jul 24 03:18:23.299910 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.299771 2579 manager.go:354] Recovery completed Jul 24 03:18:23.304228 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.304216 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 03:18:23.305989 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.305973 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-134.ec2.internal" event="NodeHasSufficientMemory" Jul 24 03:18:23.306059 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.306000 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-134.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 03:18:23.306059 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.306012 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-134.ec2.internal" event="NodeHasSufficientPID" Jul 24 03:18:23.306454 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.306439 2579 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 24 03:18:23.306454 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.306451 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 03:18:23.306550 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.306451 2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 24 03:18:23.306550 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.306516 2579 state_mem.go:36] "Initialized new in-memory state store" Jul 24 03:18:23.307055 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.307042 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-134.ec2.internal" event="NodeHasSufficientMemory" Jul 24 03:18:23.307096 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.307066 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-134.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 03:18:23.307096 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.307077 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-134.ec2.internal" event="NodeHasSufficientPID" Jul 24 03:18:23.308284 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.308273 2579 policy_none.go:49] "None policy: Start" Jul 24 03:18:23.308321 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.308288 2579 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 24 03:18:23.308321 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.308298 2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 24 03:18:23.309184 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.309173 2579 policy_none.go:47] "Start" Jul 24 03:18:23.344210 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.344197 2579 manager.go:337] "Starting Device Plugin manager" Jul 24 03:18:23.344300 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:23.344225 2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 24 03:18:23.344300 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.344234 2579 server.go:87] "Starting device plugin registration server" Jul 24 03:18:23.344449 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.344438 2579 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 24 03:18:23.344495 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.344464 2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 24 03:18:23.344639 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.344626 2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 24 03:18:23.344724 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.344712 2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 24 03:18:23.344765 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.344725 2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 24 03:18:23.345278 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:23.345262 2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 24 03:18:23.345352 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:23.345311 2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-134.ec2.internal\" not found" Jul 24 03:18:23.445174 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.445148 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 03:18:23.446283 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.446042 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-134.ec2.internal" event="NodeHasSufficientMemory" Jul 24 03:18:23.446283 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.446074 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-134.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 03:18:23.446283 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.446085 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-134.ec2.internal" event="NodeHasSufficientPID" Jul 24 03:18:23.446283 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.446124 2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-134.ec2.internal" Jul 24 03:18:23.452424 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.452407 2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-134.ec2.internal" Jul 24 03:18:23.452491 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:23.452430 2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-134.ec2.internal\": node \"ip-10-0-139-134.ec2.internal\" not found" Jul 24 03:18:23.494906 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:23.494847 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-134.ec2.internal\" not found" Jul 24 03:18:23.513701 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.513673 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 24 03:18:23.514965 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.514952 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 24 03:18:23.515038 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.514987 2579 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 24 03:18:23.515038 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.515009 2579 kubelet.go:2448] "Starting kubelet main sync loop" Jul 24 03:18:23.515125 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:23.515042 2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 24 03:18:23.517566 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.517547 2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 03:18:23.595974 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:23.595940 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-134.ec2.internal\" not found" Jul 24 03:18:23.616649 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.616609 2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-139-134.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-134.ec2.internal"] Jul 24 03:18:23.616759 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.616743 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 03:18:23.619023 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.619008 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-134.ec2.internal" event="NodeHasSufficientMemory" Jul 24 03:18:23.619086 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.619037 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-134.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 03:18:23.619086 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.619048 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-134.ec2.internal" event="NodeHasSufficientPID" Jul 24 03:18:23.621361 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.621348 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 03:18:23.621468 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.621444 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-134.ec2.internal" Jul 24 03:18:23.621518 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.621474 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 03:18:23.623728 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.623702 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-134.ec2.internal" event="NodeHasSufficientMemory" Jul 24 03:18:23.623728 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.623714 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-134.ec2.internal" event="NodeHasSufficientMemory" Jul 24 03:18:23.623860 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.623737 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-134.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 03:18:23.623860 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.623764 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-134.ec2.internal" event="NodeHasSufficientPID" Jul 24 03:18:23.623860 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.623817 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-134.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 03:18:23.623860 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.623836 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-134.ec2.internal" event="NodeHasSufficientPID" Jul 24 03:18:23.625922 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.625904 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-134.ec2.internal" Jul 24 03:18:23.625978 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.625937 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 03:18:23.627929 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.627904 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-134.ec2.internal" event="NodeHasSufficientMemory" Jul 24 03:18:23.628010 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.627934 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-134.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 03:18:23.628010 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.627946 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-134.ec2.internal" event="NodeHasSufficientPID" Jul 24 03:18:23.654379 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:23.654359 2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-134.ec2.internal\" not found" node="ip-10-0-139-134.ec2.internal" Jul 24 03:18:23.657903 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:23.657884 2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-134.ec2.internal\" not found" node="ip-10-0-139-134.ec2.internal" Jul 24 03:18:23.688585 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.688557 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0db6a30028026bdb73215df8b7b1a010-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-134.ec2.internal\" (UID: \"0db6a30028026bdb73215df8b7b1a010\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-134.ec2.internal" Jul 24 03:18:23.688693 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.688617 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0db6a30028026bdb73215df8b7b1a010-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-134.ec2.internal\" (UID: \"0db6a30028026bdb73215df8b7b1a010\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-134.ec2.internal" Jul 24 03:18:23.688693 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.688641 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0ae97c5152448a4aac7688f6e2b6cf38-config\") pod \"kube-apiserver-proxy-ip-10-0-139-134.ec2.internal\" (UID: \"0ae97c5152448a4aac7688f6e2b6cf38\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-134.ec2.internal" Jul 24 03:18:23.697109 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:23.697092 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-134.ec2.internal\" not found" Jul 24 03:18:23.788984 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.788904 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0db6a30028026bdb73215df8b7b1a010-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-134.ec2.internal\" (UID: \"0db6a30028026bdb73215df8b7b1a010\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-134.ec2.internal" Jul 24 03:18:23.788984 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.788942 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0db6a30028026bdb73215df8b7b1a010-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-134.ec2.internal\" (UID: \"0db6a30028026bdb73215df8b7b1a010\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-134.ec2.internal" Jul 24 03:18:23.788984 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.788960 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0ae97c5152448a4aac7688f6e2b6cf38-config\") pod \"kube-apiserver-proxy-ip-10-0-139-134.ec2.internal\" (UID: \"0ae97c5152448a4aac7688f6e2b6cf38\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-134.ec2.internal" Jul 24 03:18:23.789191 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.788989 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0ae97c5152448a4aac7688f6e2b6cf38-config\") pod \"kube-apiserver-proxy-ip-10-0-139-134.ec2.internal\" (UID: \"0ae97c5152448a4aac7688f6e2b6cf38\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-134.ec2.internal" Jul 24 03:18:23.789191 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.789004 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0db6a30028026bdb73215df8b7b1a010-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-134.ec2.internal\" (UID: \"0db6a30028026bdb73215df8b7b1a010\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-134.ec2.internal" Jul 24 03:18:23.789191 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.789009 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0db6a30028026bdb73215df8b7b1a010-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-134.ec2.internal\" (UID: \"0db6a30028026bdb73215df8b7b1a010\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-134.ec2.internal" Jul 24 03:18:23.798211 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:23.798187 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-134.ec2.internal\" not found" Jul 24 03:18:23.899289 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:23.899248 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-134.ec2.internal\" not found" Jul 24 03:18:23.960734 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.960692 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-134.ec2.internal" Jul 24 03:18:23.962964 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:23.962950 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-134.ec2.internal" Jul 24 03:18:23.999836 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:23.999794 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-134.ec2.internal\" not found" Jul 24 03:18:24.100042 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:24.099952 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-134.ec2.internal\" not found" Jul 24 03:18:24.160480 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.160451 2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 03:18:24.201158 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:24.201133 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-134.ec2.internal\" not found" Jul 24 03:18:24.221814 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.221796 2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 03:18:24.225936 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.225920 2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 24 03:18:24.226051 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.226034 2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 24 03:18:24.226103 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.226078 2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 24 03:18:24.226103 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.226086 2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 24 03:18:24.263170 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.263142 2579 apiserver.go:52] "Watching apiserver" Jul 24 03:18:24.286137 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.286110 2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-134.ec2.internal" Jul 24 03:18:24.286137 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.286133 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 24 03:18:24.286297 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.286150 2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 24 03:18:24.287207 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.287185 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-8hlhr","kube-system/konnectivity-agent-qmbvk","openshift-cluster-node-tuning-operator/tuned-brtgx","openshift-multus/multus-additional-cni-plugins-94swq","openshift-multus/multus-jkzg8","openshift-multus/network-metrics-daemon-gkmfm","openshift-network-diagnostics/network-check-target-52h8l","openshift-network-operator/iptables-alerter-8pzd4","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6fg2j","openshift-dns/node-resolver-gfdrz","openshift-image-registry/node-ca-5nlhv","openshift-ovn-kubernetes/ovnkube-node-ffrlv"] Jul 24 03:18:24.289727 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.289710 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:18:24.289824 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:24.289790 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hlhr" podUID="041beb02-32e9-4b8e-8f9d-34ee8d1a72c6" Jul 24 03:18:24.291777 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.291760 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qmbvk" Jul 24 03:18:24.293977 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.293959 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.294136 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.294109 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-rvhd9\"" Jul 24 03:18:24.294239 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.294146 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-23 03:13:24 +0000 UTC" deadline="2028-03-06 17:35:04.858854521 +0000 UTC" Jul 24 03:18:24.294239 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.294179 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14198h16m40.56468018s" Jul 24 03:18:24.294328 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.294281 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 24 03:18:24.294370 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.294349 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 24 03:18:24.296224 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.296206 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-94swq" Jul 24 03:18:24.296715 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.296699 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-znqwz\"" Jul 24 03:18:24.296952 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.296936 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 24 03:18:24.297481 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.297467 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 24 03:18:24.298563 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.298550 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.298778 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.298756 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 24 03:18:24.298866 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.298761 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 24 03:18:24.298866 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.298818 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 24 03:18:24.298866 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.298822 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 24 03:18:24.298990 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.298887 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zsk7h\"" Jul 24 03:18:24.298990 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.298926 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 24 03:18:24.300822 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.300805 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:18:24.300903 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:24.300861 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkmfm" podUID="1bc60e51-e1b5-4f68-a907-600eb7bc0e0b" Jul 24 03:18:24.301363 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.301347 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 24 03:18:24.301779 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.301760 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-jb982\"" Jul 24 03:18:24.303030 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.303016 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:18:24.303085 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:24.303065 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52h8l" podUID="1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85" Jul 24 03:18:24.305228 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.305210 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8pzd4" Jul 24 03:18:24.307450 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.307435 2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 24 03:18:24.307538 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.307484 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6fg2j" Jul 24 03:18:24.307630 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.307487 2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-134.ec2.internal" Jul 24 03:18:24.309395 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.309374 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 24 03:18:24.309873 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.309854 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 24 03:18:24.309989 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.309954 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gfdrz" Jul 24 03:18:24.311487 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.311472 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 24 03:18:24.311578 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.311559 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 24 03:18:24.311793 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.311778 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 24 03:18:24.311877 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.311821 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-r9zrw\"" Jul 24 03:18:24.311877 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.311826 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-4t5w2\"" Jul 24 03:18:24.311877 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.311862 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 24 03:18:24.312036 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.311904 2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 24 03:18:24.312185 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.312170 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 24 03:18:24.312243 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.312202 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 24 03:18:24.312311 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.312296 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5nlhv" Jul 24 03:18:24.312626 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.312612 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4pz46\"" Jul 24 03:18:24.315091 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.315068 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-139-134.ec2.internal"] Jul 24 03:18:24.315185 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.315156 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 24 03:18:24.315252 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.315192 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-bj64q\"" Jul 24 03:18:24.315307 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.315270 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 24 03:18:24.315307 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.315284 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.315713 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.315695 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 24 03:18:24.319458 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.319432 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 24 03:18:24.319560 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.319465 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 24 03:18:24.319560 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.319466 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 24 03:18:24.319560 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.319443 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 24 03:18:24.320127 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.320096 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 24 03:18:24.321683 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.321199 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 24 03:18:24.321683 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.321544 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-134.ec2.internal"] Jul 24 03:18:24.321892 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.321710 2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 24 03:18:24.321892 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.321792 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-clwxh\"" Jul 24 03:18:24.340842 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.340812 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-n9zr2" Jul 24 03:18:24.349664 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.349646 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-n9zr2" Jul 24 03:18:24.386924 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.386900 2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 24 03:18:24.393861 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.393842 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d97ll\" (UniqueName: \"kubernetes.io/projected/85c2fe64-4089-4a8e-b108-74936e8093ec-kube-api-access-d97ll\") pod \"iptables-alerter-8pzd4\" (UID: \"85c2fe64-4089-4a8e-b108-74936e8093ec\") " pod="openshift-network-operator/iptables-alerter-8pzd4" Jul 24 03:18:24.393984 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.393871 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d8wq\" (UniqueName: \"kubernetes.io/projected/1bc60e51-e1b5-4f68-a907-600eb7bc0e0b-kube-api-access-8d8wq\") pod \"network-metrics-daemon-gkmfm\" (UID: \"1bc60e51-e1b5-4f68-a907-600eb7bc0e0b\") " pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:18:24.393984 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.393899 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-host-kubelet\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.393984 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.393926 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6b2e05ef-119c-4450-9827-f806c437f29f-ovnkube-script-lib\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.393984 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.393953 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/47ab9dde-f95f-4503-9087-b3ba282dc067-etc-systemd\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.393984 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.393980 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-host-var-lib-cni-bin\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.394264 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394019 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-multus-conf-dir\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.394264 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394055 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/46b3c291-be47-4a57-82c2-b82e84a9909f-host\") pod \"node-ca-5nlhv\" (UID: \"46b3c291-be47-4a57-82c2-b82e84a9909f\") " pod="openshift-image-registry/node-ca-5nlhv" Jul 24 03:18:24.394264 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394073 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c4228fed-1593-4121-99cc-a5ccc6143da7-cnibin\") pod \"multus-additional-cni-plugins-94swq\" (UID: \"c4228fed-1593-4121-99cc-a5ccc6143da7\") " pod="openshift-multus/multus-additional-cni-plugins-94swq" Jul 24 03:18:24.394264 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394092 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c4228fed-1593-4121-99cc-a5ccc6143da7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-94swq\" (UID: \"c4228fed-1593-4121-99cc-a5ccc6143da7\") " pod="openshift-multus/multus-additional-cni-plugins-94swq" Jul 24 03:18:24.394264 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394109 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mn9cb\" (UniqueName: \"kubernetes.io/projected/ba246244-7a40-425a-a886-a732efbe7c24-kube-api-access-mn9cb\") pod \"aws-ebs-csi-driver-node-6fg2j\" (UID: \"ba246244-7a40-425a-a886-a732efbe7c24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6fg2j" Jul 24 03:18:24.394264 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394125 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/47ab9dde-f95f-4503-9087-b3ba282dc067-etc-sysconfig\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.394264 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394139 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/47ab9dde-f95f-4503-9087-b3ba282dc067-tmp\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.394264 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394155 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-host-run-k8s-cni-cncf-io\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.394264 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394170 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-hostroot\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.394264 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394185 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/85c2fe64-4089-4a8e-b108-74936e8093ec-iptables-alerter-script\") pod \"iptables-alerter-8pzd4\" (UID: \"85c2fe64-4089-4a8e-b108-74936e8093ec\") " pod="openshift-network-operator/iptables-alerter-8pzd4" Jul 24 03:18:24.394264 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394204 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c4228fed-1593-4121-99cc-a5ccc6143da7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-94swq\" (UID: \"c4228fed-1593-4121-99cc-a5ccc6143da7\") " pod="openshift-multus/multus-additional-cni-plugins-94swq" Jul 24 03:18:24.394264 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394221 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-host-run-netns\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.394264 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394241 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-var-lib-openvswitch\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.394264 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394260 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfqtr\" (UniqueName: \"kubernetes.io/projected/1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85-kube-api-access-qfqtr\") pod \"network-check-target-52h8l\" (UID: \"1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85\") " pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:18:24.395036 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394293 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-etc-openvswitch\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.395036 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394316 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-host-run-ovn-kubernetes\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.395036 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394340 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-host-cni-bin\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.395036 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394357 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6b2e05ef-119c-4450-9827-f806c437f29f-env-overrides\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.395036 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394372 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/47ab9dde-f95f-4503-9087-b3ba282dc067-etc-tuned\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.395036 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394420 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbmcb\" (UniqueName: \"kubernetes.io/projected/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-kube-api-access-lbmcb\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.395036 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394435 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/46b3c291-be47-4a57-82c2-b82e84a9909f-serviceca\") pod \"node-ca-5nlhv\" (UID: \"46b3c291-be47-4a57-82c2-b82e84a9909f\") " pod="openshift-image-registry/node-ca-5nlhv" Jul 24 03:18:24.395036 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394472 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c4228fed-1593-4121-99cc-a5ccc6143da7-os-release\") pod \"multus-additional-cni-plugins-94swq\" (UID: \"c4228fed-1593-4121-99cc-a5ccc6143da7\") " pod="openshift-multus/multus-additional-cni-plugins-94swq" Jul 24 03:18:24.395036 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394496 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/041beb02-32e9-4b8e-8f9d-34ee8d1a72c6-kubelet-config\") pod \"global-pull-secret-syncer-8hlhr\" (UID: \"041beb02-32e9-4b8e-8f9d-34ee8d1a72c6\") " pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:18:24.395036 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394511 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/47ab9dde-f95f-4503-9087-b3ba282dc067-lib-modules\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.395036 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394539 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-cni-binary-copy\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.395036 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394573 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-multus-socket-dir-parent\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.395036 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394614 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-run-openvswitch\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.395036 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394649 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-node-log\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.395036 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394706 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/041beb02-32e9-4b8e-8f9d-34ee8d1a72c6-dbus\") pod \"global-pull-secret-syncer-8hlhr\" (UID: \"041beb02-32e9-4b8e-8f9d-34ee8d1a72c6\") " pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:18:24.395036 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394727 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snpr5\" (UniqueName: \"kubernetes.io/projected/46b3c291-be47-4a57-82c2-b82e84a9909f-kube-api-access-snpr5\") pod \"node-ca-5nlhv\" (UID: \"46b3c291-be47-4a57-82c2-b82e84a9909f\") " pod="openshift-image-registry/node-ca-5nlhv" Jul 24 03:18:24.395892 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394742 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/47ab9dde-f95f-4503-9087-b3ba282dc067-etc-sysctl-d\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.395892 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394763 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c2c5797b-2cf9-4743-b334-7efa2192095a-agent-certs\") pod \"konnectivity-agent-qmbvk\" (UID: \"c2c5797b-2cf9-4743-b334-7efa2192095a\") " pod="kube-system/konnectivity-agent-qmbvk" Jul 24 03:18:24.395892 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394783 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c2c5797b-2cf9-4743-b334-7efa2192095a-konnectivity-ca\") pod \"konnectivity-agent-qmbvk\" (UID: \"c2c5797b-2cf9-4743-b334-7efa2192095a\") " pod="kube-system/konnectivity-agent-qmbvk" Jul 24 03:18:24.395892 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394799 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq4jt\" (UniqueName: \"kubernetes.io/projected/6b2e05ef-119c-4450-9827-f806c437f29f-kube-api-access-zq4jt\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.395892 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394815 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c4228fed-1593-4121-99cc-a5ccc6143da7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-94swq\" (UID: \"c4228fed-1593-4121-99cc-a5ccc6143da7\") " pod="openshift-multus/multus-additional-cni-plugins-94swq" Jul 24 03:18:24.395892 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394836 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fc751d2-6a75-413f-881c-5a3c92935de3-tmp-dir\") pod \"node-resolver-gfdrz\" (UID: \"5fc751d2-6a75-413f-881c-5a3c92935de3\") " pod="openshift-dns/node-resolver-gfdrz" Jul 24 03:18:24.395892 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394862 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6b2e05ef-119c-4450-9827-f806c437f29f-ovn-node-metrics-cert\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.395892 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394905 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5fc751d2-6a75-413f-881c-5a3c92935de3-hosts-file\") pod \"node-resolver-gfdrz\" (UID: \"5fc751d2-6a75-413f-881c-5a3c92935de3\") " pod="openshift-dns/node-resolver-gfdrz" Jul 24 03:18:24.395892 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394929 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.395892 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394954 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-os-release\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.395892 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.394984 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-host-var-lib-kubelet\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.395892 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.395015 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-etc-kubernetes\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.395892 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.395086 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4g8f\" (UniqueName: \"kubernetes.io/projected/c4228fed-1593-4121-99cc-a5ccc6143da7-kube-api-access-g4g8f\") pod \"multus-additional-cni-plugins-94swq\" (UID: \"c4228fed-1593-4121-99cc-a5ccc6143da7\") " pod="openshift-multus/multus-additional-cni-plugins-94swq" Jul 24 03:18:24.395892 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.395114 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-log-socket\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.395892 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.395128 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-host-cni-netd\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.395892 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.395146 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/47ab9dde-f95f-4503-9087-b3ba282dc067-etc-sysctl-conf\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.396785 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.395188 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/47ab9dde-f95f-4503-9087-b3ba282dc067-run\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.396785 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.395213 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/47ab9dde-f95f-4503-9087-b3ba282dc067-sys\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.396785 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.395238 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-system-cni-dir\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.396785 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.395263 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-host-var-lib-cni-multus\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.396785 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.395303 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c4228fed-1593-4121-99cc-a5ccc6143da7-system-cni-dir\") pod \"multus-additional-cni-plugins-94swq\" (UID: \"c4228fed-1593-4121-99cc-a5ccc6143da7\") " pod="openshift-multus/multus-additional-cni-plugins-94swq" Jul 24 03:18:24.396785 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.395340 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ba246244-7a40-425a-a886-a732efbe7c24-registration-dir\") pod \"aws-ebs-csi-driver-node-6fg2j\" (UID: \"ba246244-7a40-425a-a886-a732efbe7c24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6fg2j" Jul 24 03:18:24.396785 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.395372 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/47ab9dde-f95f-4503-9087-b3ba282dc067-var-lib-kubelet\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.396785 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.395435 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-host-run-netns\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.396785 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.395470 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-multus-daemon-config\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.396785 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.395568 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-host-run-multus-certs\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.396785 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.395592 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c4228fed-1593-4121-99cc-a5ccc6143da7-cni-binary-copy\") pod \"multus-additional-cni-plugins-94swq\" (UID: \"c4228fed-1593-4121-99cc-a5ccc6143da7\") " pod="openshift-multus/multus-additional-cni-plugins-94swq" Jul 24 03:18:24.396785 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.395634 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ba246244-7a40-425a-a886-a732efbe7c24-device-dir\") pod \"aws-ebs-csi-driver-node-6fg2j\" (UID: \"ba246244-7a40-425a-a886-a732efbe7c24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6fg2j" Jul 24 03:18:24.396785 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.395688 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ba246244-7a40-425a-a886-a732efbe7c24-sys-fs\") pod \"aws-ebs-csi-driver-node-6fg2j\" (UID: \"ba246244-7a40-425a-a886-a732efbe7c24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6fg2j" Jul 24 03:18:24.396785 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.395720 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-cnibin\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.396785 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.395739 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/85c2fe64-4089-4a8e-b108-74936e8093ec-host-slash\") pod \"iptables-alerter-8pzd4\" (UID: \"85c2fe64-4089-4a8e-b108-74936e8093ec\") " pod="openshift-network-operator/iptables-alerter-8pzd4" Jul 24 03:18:24.396785 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.395767 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bc60e51-e1b5-4f68-a907-600eb7bc0e0b-metrics-certs\") pod \"network-metrics-daemon-gkmfm\" (UID: \"1bc60e51-e1b5-4f68-a907-600eb7bc0e0b\") " pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:18:24.397325 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.395791 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2wwr\" (UniqueName: \"kubernetes.io/projected/5fc751d2-6a75-413f-881c-5a3c92935de3-kube-api-access-v2wwr\") pod \"node-resolver-gfdrz\" (UID: \"5fc751d2-6a75-413f-881c-5a3c92935de3\") " pod="openshift-dns/node-resolver-gfdrz" Jul 24 03:18:24.397325 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.395818 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-systemd-units\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.397325 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.395867 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-run-systemd\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.397325 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.395893 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-run-ovn\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.397325 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.395909 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6b2e05ef-119c-4450-9827-f806c437f29f-ovnkube-config\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.397325 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.395927 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/47ab9dde-f95f-4503-9087-b3ba282dc067-etc-modprobe-d\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.397325 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.395950 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47ab9dde-f95f-4503-9087-b3ba282dc067-etc-kubernetes\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.397325 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.395970 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/47ab9dde-f95f-4503-9087-b3ba282dc067-host\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.397325 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.395986 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-multus-cni-dir\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.397325 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.396019 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ba246244-7a40-425a-a886-a732efbe7c24-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6fg2j\" (UID: \"ba246244-7a40-425a-a886-a732efbe7c24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6fg2j" Jul 24 03:18:24.397325 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.396048 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ba246244-7a40-425a-a886-a732efbe7c24-socket-dir\") pod \"aws-ebs-csi-driver-node-6fg2j\" (UID: \"ba246244-7a40-425a-a886-a732efbe7c24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6fg2j" Jul 24 03:18:24.397325 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.396067 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ba246244-7a40-425a-a886-a732efbe7c24-etc-selinux\") pod \"aws-ebs-csi-driver-node-6fg2j\" (UID: \"ba246244-7a40-425a-a886-a732efbe7c24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6fg2j" Jul 24 03:18:24.397325 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.396099 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-host-slash\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.397325 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.396155 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/041beb02-32e9-4b8e-8f9d-34ee8d1a72c6-original-pull-secret\") pod \"global-pull-secret-syncer-8hlhr\" (UID: \"041beb02-32e9-4b8e-8f9d-34ee8d1a72c6\") " pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:18:24.397325 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.396196 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwgnq\" (UniqueName: \"kubernetes.io/projected/47ab9dde-f95f-4503-9087-b3ba282dc067-kube-api-access-bwgnq\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.496777 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.496743 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c4228fed-1593-4121-99cc-a5ccc6143da7-system-cni-dir\") pod \"multus-additional-cni-plugins-94swq\" (UID: \"c4228fed-1593-4121-99cc-a5ccc6143da7\") " pod="openshift-multus/multus-additional-cni-plugins-94swq" Jul 24 03:18:24.496777 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.496775 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ba246244-7a40-425a-a886-a732efbe7c24-registration-dir\") pod \"aws-ebs-csi-driver-node-6fg2j\" (UID: \"ba246244-7a40-425a-a886-a732efbe7c24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6fg2j" Jul 24 03:18:24.497052 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.496797 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/47ab9dde-f95f-4503-9087-b3ba282dc067-var-lib-kubelet\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.497052 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.496821 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-host-run-netns\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.497052 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.496860 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-multus-daemon-config\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.497052 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.496886 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-host-run-multus-certs\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.497052 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.496914 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c4228fed-1593-4121-99cc-a5ccc6143da7-cni-binary-copy\") pod \"multus-additional-cni-plugins-94swq\" (UID: \"c4228fed-1593-4121-99cc-a5ccc6143da7\") " pod="openshift-multus/multus-additional-cni-plugins-94swq" Jul 24 03:18:24.497052 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.496861 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c4228fed-1593-4121-99cc-a5ccc6143da7-system-cni-dir\") pod \"multus-additional-cni-plugins-94swq\" (UID: \"c4228fed-1593-4121-99cc-a5ccc6143da7\") " pod="openshift-multus/multus-additional-cni-plugins-94swq" Jul 24 03:18:24.497052 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.496930 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ba246244-7a40-425a-a886-a732efbe7c24-registration-dir\") pod \"aws-ebs-csi-driver-node-6fg2j\" (UID: \"ba246244-7a40-425a-a886-a732efbe7c24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6fg2j" Jul 24 03:18:24.497052 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.496932 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ba246244-7a40-425a-a886-a732efbe7c24-device-dir\") pod \"aws-ebs-csi-driver-node-6fg2j\" (UID: \"ba246244-7a40-425a-a886-a732efbe7c24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6fg2j" Jul 24 03:18:24.497052 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.496979 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ba246244-7a40-425a-a886-a732efbe7c24-sys-fs\") pod \"aws-ebs-csi-driver-node-6fg2j\" (UID: \"ba246244-7a40-425a-a886-a732efbe7c24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6fg2j" Jul 24 03:18:24.497052 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.496972 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-host-run-multus-certs\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.497052 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.497003 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-cnibin\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.497052 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.496990 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ba246244-7a40-425a-a886-a732efbe7c24-device-dir\") pod \"aws-ebs-csi-driver-node-6fg2j\" (UID: \"ba246244-7a40-425a-a886-a732efbe7c24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6fg2j" Jul 24 03:18:24.497052 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.496941 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/47ab9dde-f95f-4503-9087-b3ba282dc067-var-lib-kubelet\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.497052 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.496930 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-host-run-netns\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.497052 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.497034 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/85c2fe64-4089-4a8e-b108-74936e8093ec-host-slash\") pod \"iptables-alerter-8pzd4\" (UID: \"85c2fe64-4089-4a8e-b108-74936e8093ec\") " pod="openshift-network-operator/iptables-alerter-8pzd4" Jul 24 03:18:24.497703 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.497068 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/85c2fe64-4089-4a8e-b108-74936e8093ec-host-slash\") pod \"iptables-alerter-8pzd4\" (UID: \"85c2fe64-4089-4a8e-b108-74936e8093ec\") " pod="openshift-network-operator/iptables-alerter-8pzd4" Jul 24 03:18:24.497703 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.497078 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bc60e51-e1b5-4f68-a907-600eb7bc0e0b-metrics-certs\") pod \"network-metrics-daemon-gkmfm\" (UID: \"1bc60e51-e1b5-4f68-a907-600eb7bc0e0b\") " pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:18:24.497703 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.497068 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ba246244-7a40-425a-a886-a732efbe7c24-sys-fs\") pod \"aws-ebs-csi-driver-node-6fg2j\" (UID: \"ba246244-7a40-425a-a886-a732efbe7c24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6fg2j" Jul 24 03:18:24.497703 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.497090 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-cnibin\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.497703 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.497170 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2wwr\" (UniqueName: \"kubernetes.io/projected/5fc751d2-6a75-413f-881c-5a3c92935de3-kube-api-access-v2wwr\") pod \"node-resolver-gfdrz\" (UID: \"5fc751d2-6a75-413f-881c-5a3c92935de3\") " pod="openshift-dns/node-resolver-gfdrz" Jul 24 03:18:24.497703 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:24.497274 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 03:18:24.497703 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.497330 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-systemd-units\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.497703 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.497376 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-systemd-units\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.497703 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:24.497381 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bc60e51-e1b5-4f68-a907-600eb7bc0e0b-metrics-certs podName:1bc60e51-e1b5-4f68-a907-600eb7bc0e0b nodeName:}" failed. No retries permitted until 2026-07-24 03:18:24.99733369 +0000 UTC m=+2.272905036 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bc60e51-e1b5-4f68-a907-600eb7bc0e0b-metrics-certs") pod "network-metrics-daemon-gkmfm" (UID: "1bc60e51-e1b5-4f68-a907-600eb7bc0e0b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 03:18:24.497703 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.497425 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-run-systemd\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.497703 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.497452 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-run-ovn\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.497703 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.497479 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6b2e05ef-119c-4450-9827-f806c437f29f-ovnkube-config\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.497703 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.497491 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-run-systemd\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.497703 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.497509 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/47ab9dde-f95f-4503-9087-b3ba282dc067-etc-modprobe-d\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.497703 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.497515 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c4228fed-1593-4121-99cc-a5ccc6143da7-cni-binary-copy\") pod \"multus-additional-cni-plugins-94swq\" (UID: \"c4228fed-1593-4121-99cc-a5ccc6143da7\") " pod="openshift-multus/multus-additional-cni-plugins-94swq" Jul 24 03:18:24.497703 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.497534 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47ab9dde-f95f-4503-9087-b3ba282dc067-etc-kubernetes\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.497703 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.497544 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-run-ovn\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.498664 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.497525 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-multus-daemon-config\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.498664 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.497873 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/47ab9dde-f95f-4503-9087-b3ba282dc067-etc-modprobe-d\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.498664 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.498102 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47ab9dde-f95f-4503-9087-b3ba282dc067-etc-kubernetes\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.498664 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.498399 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/47ab9dde-f95f-4503-9087-b3ba282dc067-host\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.499074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.498900 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/47ab9dde-f95f-4503-9087-b3ba282dc067-host\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.499074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.498956 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-multus-cni-dir\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.499074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.498994 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ba246244-7a40-425a-a886-a732efbe7c24-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6fg2j\" (UID: \"ba246244-7a40-425a-a886-a732efbe7c24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6fg2j" Jul 24 03:18:24.499074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.499026 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ba246244-7a40-425a-a886-a732efbe7c24-socket-dir\") pod \"aws-ebs-csi-driver-node-6fg2j\" (UID: \"ba246244-7a40-425a-a886-a732efbe7c24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6fg2j" Jul 24 03:18:24.499074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.499067 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ba246244-7a40-425a-a886-a732efbe7c24-etc-selinux\") pod \"aws-ebs-csi-driver-node-6fg2j\" (UID: \"ba246244-7a40-425a-a886-a732efbe7c24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6fg2j" Jul 24 03:18:24.499351 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.499104 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-host-slash\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.499351 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.499130 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/041beb02-32e9-4b8e-8f9d-34ee8d1a72c6-original-pull-secret\") pod \"global-pull-secret-syncer-8hlhr\" (UID: \"041beb02-32e9-4b8e-8f9d-34ee8d1a72c6\") " pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:18:24.499351 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.499160 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwgnq\" (UniqueName: \"kubernetes.io/projected/47ab9dde-f95f-4503-9087-b3ba282dc067-kube-api-access-bwgnq\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.499351 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.499191 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d97ll\" (UniqueName: \"kubernetes.io/projected/85c2fe64-4089-4a8e-b108-74936e8093ec-kube-api-access-d97ll\") pod \"iptables-alerter-8pzd4\" (UID: \"85c2fe64-4089-4a8e-b108-74936e8093ec\") " pod="openshift-network-operator/iptables-alerter-8pzd4" Jul 24 03:18:24.499351 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.499224 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8d8wq\" (UniqueName: \"kubernetes.io/projected/1bc60e51-e1b5-4f68-a907-600eb7bc0e0b-kube-api-access-8d8wq\") pod \"network-metrics-daemon-gkmfm\" (UID: \"1bc60e51-e1b5-4f68-a907-600eb7bc0e0b\") " pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:18:24.499351 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.499257 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-host-kubelet\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.499351 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.499283 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6b2e05ef-119c-4450-9827-f806c437f29f-ovnkube-script-lib\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.499351 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.499319 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/47ab9dde-f95f-4503-9087-b3ba282dc067-etc-systemd\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.499351 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.499348 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-host-var-lib-cni-bin\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.499789 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.499413 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-multus-conf-dir\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.499789 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.499446 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/46b3c291-be47-4a57-82c2-b82e84a9909f-host\") pod \"node-ca-5nlhv\" (UID: \"46b3c291-be47-4a57-82c2-b82e84a9909f\") " pod="openshift-image-registry/node-ca-5nlhv" Jul 24 03:18:24.499789 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.499483 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c4228fed-1593-4121-99cc-a5ccc6143da7-cnibin\") pod \"multus-additional-cni-plugins-94swq\" (UID: \"c4228fed-1593-4121-99cc-a5ccc6143da7\") " pod="openshift-multus/multus-additional-cni-plugins-94swq" Jul 24 03:18:24.499789 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.499511 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c4228fed-1593-4121-99cc-a5ccc6143da7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-94swq\" (UID: \"c4228fed-1593-4121-99cc-a5ccc6143da7\") " pod="openshift-multus/multus-additional-cni-plugins-94swq" Jul 24 03:18:24.499789 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.499567 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mn9cb\" (UniqueName: \"kubernetes.io/projected/ba246244-7a40-425a-a886-a732efbe7c24-kube-api-access-mn9cb\") pod \"aws-ebs-csi-driver-node-6fg2j\" (UID: \"ba246244-7a40-425a-a886-a732efbe7c24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6fg2j" Jul 24 03:18:24.499789 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.499599 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/47ab9dde-f95f-4503-9087-b3ba282dc067-etc-sysconfig\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.499789 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.499627 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/47ab9dde-f95f-4503-9087-b3ba282dc067-tmp\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.499789 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.499653 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-host-run-k8s-cni-cncf-io\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.499789 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.499700 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-hostroot\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.499789 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.499733 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/85c2fe64-4089-4a8e-b108-74936e8093ec-iptables-alerter-script\") pod \"iptables-alerter-8pzd4\" (UID: \"85c2fe64-4089-4a8e-b108-74936e8093ec\") " pod="openshift-network-operator/iptables-alerter-8pzd4" Jul 24 03:18:24.499789 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.499770 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c4228fed-1593-4121-99cc-a5ccc6143da7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-94swq\" (UID: \"c4228fed-1593-4121-99cc-a5ccc6143da7\") " pod="openshift-multus/multus-additional-cni-plugins-94swq" Jul 24 03:18:24.500379 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.499803 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-host-run-netns\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.500379 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.499828 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-var-lib-openvswitch\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.500379 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.499860 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfqtr\" (UniqueName: \"kubernetes.io/projected/1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85-kube-api-access-qfqtr\") pod \"network-check-target-52h8l\" (UID: \"1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85\") " pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:18:24.500379 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.499890 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-etc-openvswitch\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.500379 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.499920 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-host-run-ovn-kubernetes\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.500379 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.499950 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-host-cni-bin\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.500379 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.499975 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6b2e05ef-119c-4450-9827-f806c437f29f-env-overrides\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.500379 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.500012 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/47ab9dde-f95f-4503-9087-b3ba282dc067-etc-tuned\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.500379 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.500037 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbmcb\" (UniqueName: \"kubernetes.io/projected/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-kube-api-access-lbmcb\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.500379 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.500072 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/46b3c291-be47-4a57-82c2-b82e84a9909f-serviceca\") pod \"node-ca-5nlhv\" (UID: \"46b3c291-be47-4a57-82c2-b82e84a9909f\") " pod="openshift-image-registry/node-ca-5nlhv" Jul 24 03:18:24.500379 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.500102 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c4228fed-1593-4121-99cc-a5ccc6143da7-os-release\") pod \"multus-additional-cni-plugins-94swq\" (UID: \"c4228fed-1593-4121-99cc-a5ccc6143da7\") " pod="openshift-multus/multus-additional-cni-plugins-94swq" Jul 24 03:18:24.500379 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.500133 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/041beb02-32e9-4b8e-8f9d-34ee8d1a72c6-kubelet-config\") pod \"global-pull-secret-syncer-8hlhr\" (UID: \"041beb02-32e9-4b8e-8f9d-34ee8d1a72c6\") " pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:18:24.500379 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.500160 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/47ab9dde-f95f-4503-9087-b3ba282dc067-lib-modules\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.500379 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.500193 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-cni-binary-copy\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.500379 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.500224 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-multus-socket-dir-parent\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.500379 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.500259 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-run-openvswitch\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.500379 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.500290 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-node-log\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.501246 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.500316 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/041beb02-32e9-4b8e-8f9d-34ee8d1a72c6-dbus\") pod \"global-pull-secret-syncer-8hlhr\" (UID: \"041beb02-32e9-4b8e-8f9d-34ee8d1a72c6\") " pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:18:24.501246 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.500347 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snpr5\" (UniqueName: \"kubernetes.io/projected/46b3c291-be47-4a57-82c2-b82e84a9909f-kube-api-access-snpr5\") pod \"node-ca-5nlhv\" (UID: \"46b3c291-be47-4a57-82c2-b82e84a9909f\") " pod="openshift-image-registry/node-ca-5nlhv" Jul 24 03:18:24.501246 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.500377 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/47ab9dde-f95f-4503-9087-b3ba282dc067-etc-sysctl-d\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.501246 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.500407 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c2c5797b-2cf9-4743-b334-7efa2192095a-agent-certs\") pod \"konnectivity-agent-qmbvk\" (UID: \"c2c5797b-2cf9-4743-b334-7efa2192095a\") " pod="kube-system/konnectivity-agent-qmbvk" Jul 24 03:18:24.501246 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.500437 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c2c5797b-2cf9-4743-b334-7efa2192095a-konnectivity-ca\") pod \"konnectivity-agent-qmbvk\" (UID: \"c2c5797b-2cf9-4743-b334-7efa2192095a\") " pod="kube-system/konnectivity-agent-qmbvk" Jul 24 03:18:24.501246 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.500463 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq4jt\" (UniqueName: \"kubernetes.io/projected/6b2e05ef-119c-4450-9827-f806c437f29f-kube-api-access-zq4jt\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.501246 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.500495 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c4228fed-1593-4121-99cc-a5ccc6143da7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-94swq\" (UID: \"c4228fed-1593-4121-99cc-a5ccc6143da7\") " pod="openshift-multus/multus-additional-cni-plugins-94swq" Jul 24 03:18:24.501246 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.500527 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fc751d2-6a75-413f-881c-5a3c92935de3-tmp-dir\") pod \"node-resolver-gfdrz\" (UID: \"5fc751d2-6a75-413f-881c-5a3c92935de3\") " pod="openshift-dns/node-resolver-gfdrz" Jul 24 03:18:24.501246 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.500567 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6b2e05ef-119c-4450-9827-f806c437f29f-ovn-node-metrics-cert\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.501246 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.500592 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5fc751d2-6a75-413f-881c-5a3c92935de3-hosts-file\") pod \"node-resolver-gfdrz\" (UID: \"5fc751d2-6a75-413f-881c-5a3c92935de3\") " pod="openshift-dns/node-resolver-gfdrz" Jul 24 03:18:24.501246 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.500630 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.501246 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.500663 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-os-release\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.501246 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.500695 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-host-var-lib-kubelet\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.501246 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.500724 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-etc-kubernetes\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.501246 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.500768 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4g8f\" (UniqueName: \"kubernetes.io/projected/c4228fed-1593-4121-99cc-a5ccc6143da7-kube-api-access-g4g8f\") pod \"multus-additional-cni-plugins-94swq\" (UID: \"c4228fed-1593-4121-99cc-a5ccc6143da7\") " pod="openshift-multus/multus-additional-cni-plugins-94swq" Jul 24 03:18:24.501246 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.500799 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-log-socket\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.501246 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.500829 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-host-cni-netd\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.502127 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.500868 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/47ab9dde-f95f-4503-9087-b3ba282dc067-etc-sysctl-conf\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.502127 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.500898 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/47ab9dde-f95f-4503-9087-b3ba282dc067-run\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.502127 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.500987 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-host-cni-bin\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.502127 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.501012 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.502127 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.501033 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/47ab9dde-f95f-4503-9087-b3ba282dc067-etc-sysctl-d\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.502127 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.500933 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-host-run-ovn-kubernetes\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.502127 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.501038 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/47ab9dde-f95f-4503-9087-b3ba282dc067-sys\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.502127 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.501204 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-host-cni-netd\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.502127 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.501205 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-host-run-k8s-cni-cncf-io\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.502127 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.501234 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-host-var-lib-kubelet\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.502127 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.501243 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-host-slash\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.502127 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.501323 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c4228fed-1593-4121-99cc-a5ccc6143da7-os-release\") pod \"multus-additional-cni-plugins-94swq\" (UID: \"c4228fed-1593-4121-99cc-a5ccc6143da7\") " pod="openshift-multus/multus-additional-cni-plugins-94swq" Jul 24 03:18:24.502127 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.501335 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-etc-kubernetes\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.502127 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.501343 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-multus-cni-dir\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.502127 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.501350 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/47ab9dde-f95f-4503-9087-b3ba282dc067-etc-sysctl-conf\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.502127 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.501356 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6b2e05ef-119c-4450-9827-f806c437f29f-ovnkube-config\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.502127 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.501361 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/47ab9dde-f95f-4503-9087-b3ba282dc067-lib-modules\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.502127 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.501421 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ba246244-7a40-425a-a886-a732efbe7c24-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6fg2j\" (UID: \"ba246244-7a40-425a-a886-a732efbe7c24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6fg2j" Jul 24 03:18:24.503084 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.501438 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/47ab9dde-f95f-4503-9087-b3ba282dc067-etc-systemd\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.503084 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.501492 2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 24 03:18:24.503084 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.501534 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/041beb02-32e9-4b8e-8f9d-34ee8d1a72c6-dbus\") pod \"global-pull-secret-syncer-8hlhr\" (UID: \"041beb02-32e9-4b8e-8f9d-34ee8d1a72c6\") " pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:18:24.503084 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.501549 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-multus-socket-dir-parent\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.503084 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.501555 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5fc751d2-6a75-413f-881c-5a3c92935de3-hosts-file\") pod \"node-resolver-gfdrz\" (UID: \"5fc751d2-6a75-413f-881c-5a3c92935de3\") " pod="openshift-dns/node-resolver-gfdrz" Jul 24 03:18:24.503084 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.501605 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-system-cni-dir\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.503084 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.501614 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-run-openvswitch\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.503084 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.501618 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/041beb02-32e9-4b8e-8f9d-34ee8d1a72c6-kubelet-config\") pod \"global-pull-secret-syncer-8hlhr\" (UID: \"041beb02-32e9-4b8e-8f9d-34ee8d1a72c6\") " pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:18:24.503084 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.501637 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/47ab9dde-f95f-4503-9087-b3ba282dc067-run\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.503084 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.501655 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-host-var-lib-cni-multus\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.503084 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.501729 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-hostroot\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.503084 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.501821 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ba246244-7a40-425a-a886-a732efbe7c24-etc-selinux\") pod \"aws-ebs-csi-driver-node-6fg2j\" (UID: \"ba246244-7a40-425a-a886-a732efbe7c24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6fg2j" Jul 24 03:18:24.503084 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.501868 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-node-log\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.503084 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.501935 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-host-run-netns\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.503084 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.502031 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-host-kubelet\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.503084 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.502124 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-host-var-lib-cni-multus\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.503084 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.502161 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6b2e05ef-119c-4450-9827-f806c437f29f-ovnkube-script-lib\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.503084 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.502164 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-multus-conf-dir\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.503843 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.502213 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c4228fed-1593-4121-99cc-a5ccc6143da7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-94swq\" (UID: \"c4228fed-1593-4121-99cc-a5ccc6143da7\") " pod="openshift-multus/multus-additional-cni-plugins-94swq" Jul 24 03:18:24.503843 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.502226 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-os-release\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.503843 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.502385 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-cni-binary-copy\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.503843 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.502434 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-var-lib-openvswitch\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.503843 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.502464 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-host-var-lib-cni-bin\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.503843 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.502496 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-etc-openvswitch\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.503843 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.502529 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/46b3c291-be47-4a57-82c2-b82e84a9909f-serviceca\") pod \"node-ca-5nlhv\" (UID: \"46b3c291-be47-4a57-82c2-b82e84a9909f\") " pod="openshift-image-registry/node-ca-5nlhv" Jul 24 03:18:24.503843 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:24.502658 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 03:18:24.503843 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.502709 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ba246244-7a40-425a-a886-a732efbe7c24-socket-dir\") pod \"aws-ebs-csi-driver-node-6fg2j\" (UID: \"ba246244-7a40-425a-a886-a732efbe7c24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6fg2j" Jul 24 03:18:24.503843 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:24.502759 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/041beb02-32e9-4b8e-8f9d-34ee8d1a72c6-original-pull-secret podName:041beb02-32e9-4b8e-8f9d-34ee8d1a72c6 nodeName:}" failed. No retries permitted until 2026-07-24 03:18:25.002716802 +0000 UTC m=+2.278288147 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/041beb02-32e9-4b8e-8f9d-34ee8d1a72c6-original-pull-secret") pod "global-pull-secret-syncer-8hlhr" (UID: "041beb02-32e9-4b8e-8f9d-34ee8d1a72c6") : object "kube-system"/"original-pull-secret" not registered Jul 24 03:18:24.503843 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.502813 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-system-cni-dir\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.503843 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.502866 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/46b3c291-be47-4a57-82c2-b82e84a9909f-host\") pod \"node-ca-5nlhv\" (UID: \"46b3c291-be47-4a57-82c2-b82e84a9909f\") " pod="openshift-image-registry/node-ca-5nlhv" Jul 24 03:18:24.503843 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.502915 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/47ab9dde-f95f-4503-9087-b3ba282dc067-sys\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.503843 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.502981 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c4228fed-1593-4121-99cc-a5ccc6143da7-cnibin\") pod \"multus-additional-cni-plugins-94swq\" (UID: \"c4228fed-1593-4121-99cc-a5ccc6143da7\") " pod="openshift-multus/multus-additional-cni-plugins-94swq" Jul 24 03:18:24.503843 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.502992 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6b2e05ef-119c-4450-9827-f806c437f29f-env-overrides\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.503843 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.503044 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6b2e05ef-119c-4450-9827-f806c437f29f-log-socket\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.503843 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.503263 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fc751d2-6a75-413f-881c-5a3c92935de3-tmp-dir\") pod \"node-resolver-gfdrz\" (UID: \"5fc751d2-6a75-413f-881c-5a3c92935de3\") " pod="openshift-dns/node-resolver-gfdrz" Jul 24 03:18:24.504824 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.504798 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/85c2fe64-4089-4a8e-b108-74936e8093ec-iptables-alerter-script\") pod \"iptables-alerter-8pzd4\" (UID: \"85c2fe64-4089-4a8e-b108-74936e8093ec\") " pod="openshift-network-operator/iptables-alerter-8pzd4" Jul 24 03:18:24.504904 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.504827 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c4228fed-1593-4121-99cc-a5ccc6143da7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-94swq\" (UID: \"c4228fed-1593-4121-99cc-a5ccc6143da7\") " pod="openshift-multus/multus-additional-cni-plugins-94swq" Jul 24 03:18:24.504904 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.504861 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/47ab9dde-f95f-4503-9087-b3ba282dc067-etc-sysconfig\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.504904 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.504879 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c2c5797b-2cf9-4743-b334-7efa2192095a-konnectivity-ca\") pod \"konnectivity-agent-qmbvk\" (UID: \"c2c5797b-2cf9-4743-b334-7efa2192095a\") " pod="kube-system/konnectivity-agent-qmbvk" Jul 24 03:18:24.505559 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.505537 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/47ab9dde-f95f-4503-9087-b3ba282dc067-etc-tuned\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.505652 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.505613 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6b2e05ef-119c-4450-9827-f806c437f29f-ovn-node-metrics-cert\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.505795 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.505772 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c2c5797b-2cf9-4743-b334-7efa2192095a-agent-certs\") pod \"konnectivity-agent-qmbvk\" (UID: \"c2c5797b-2cf9-4743-b334-7efa2192095a\") " pod="kube-system/konnectivity-agent-qmbvk" Jul 24 03:18:24.505987 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.505966 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/47ab9dde-f95f-4503-9087-b3ba282dc067-tmp\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.506087 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.506064 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c4228fed-1593-4121-99cc-a5ccc6143da7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-94swq\" (UID: \"c4228fed-1593-4121-99cc-a5ccc6143da7\") " pod="openshift-multus/multus-additional-cni-plugins-94swq" Jul 24 03:18:24.507727 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.507706 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2wwr\" (UniqueName: \"kubernetes.io/projected/5fc751d2-6a75-413f-881c-5a3c92935de3-kube-api-access-v2wwr\") pod \"node-resolver-gfdrz\" (UID: \"5fc751d2-6a75-413f-881c-5a3c92935de3\") " pod="openshift-dns/node-resolver-gfdrz" Jul 24 03:18:24.511453 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.511432 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwgnq\" (UniqueName: \"kubernetes.io/projected/47ab9dde-f95f-4503-9087-b3ba282dc067-kube-api-access-bwgnq\") pod \"tuned-brtgx\" (UID: \"47ab9dde-f95f-4503-9087-b3ba282dc067\") " pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.511811 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.511793 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mn9cb\" (UniqueName: \"kubernetes.io/projected/ba246244-7a40-425a-a886-a732efbe7c24-kube-api-access-mn9cb\") pod \"aws-ebs-csi-driver-node-6fg2j\" (UID: \"ba246244-7a40-425a-a886-a732efbe7c24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6fg2j" Jul 24 03:18:24.512259 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:24.512243 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 03:18:24.512333 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:24.512263 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 03:18:24.512333 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:24.512273 2579 projected.go:196] Error preparing data for projected volume kube-api-access-qfqtr for pod openshift-network-diagnostics/network-check-target-52h8l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 03:18:24.512333 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:24.512322 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85-kube-api-access-qfqtr podName:1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85 nodeName:}" failed. No retries permitted until 2026-07-24 03:18:25.012310575 +0000 UTC m=+2.287881905 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qfqtr" (UniqueName: "kubernetes.io/projected/1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85-kube-api-access-qfqtr") pod "network-check-target-52h8l" (UID: "1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 03:18:24.514809 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.514789 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq4jt\" (UniqueName: \"kubernetes.io/projected/6b2e05ef-119c-4450-9827-f806c437f29f-kube-api-access-zq4jt\") pod \"ovnkube-node-ffrlv\" (UID: \"6b2e05ef-119c-4450-9827-f806c437f29f\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.514950 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.514929 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4g8f\" (UniqueName: \"kubernetes.io/projected/c4228fed-1593-4121-99cc-a5ccc6143da7-kube-api-access-g4g8f\") pod \"multus-additional-cni-plugins-94swq\" (UID: \"c4228fed-1593-4121-99cc-a5ccc6143da7\") " pod="openshift-multus/multus-additional-cni-plugins-94swq" Jul 24 03:18:24.515231 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.515213 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d97ll\" (UniqueName: \"kubernetes.io/projected/85c2fe64-4089-4a8e-b108-74936e8093ec-kube-api-access-d97ll\") pod \"iptables-alerter-8pzd4\" (UID: \"85c2fe64-4089-4a8e-b108-74936e8093ec\") " pod="openshift-network-operator/iptables-alerter-8pzd4" Jul 24 03:18:24.515601 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.515583 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbmcb\" (UniqueName: \"kubernetes.io/projected/fd59f65d-aa8f-4dfb-86fa-37c8523647ce-kube-api-access-lbmcb\") pod \"multus-jkzg8\" (UID: \"fd59f65d-aa8f-4dfb-86fa-37c8523647ce\") " pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.515649 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.515607 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snpr5\" (UniqueName: \"kubernetes.io/projected/46b3c291-be47-4a57-82c2-b82e84a9909f-kube-api-access-snpr5\") pod \"node-ca-5nlhv\" (UID: \"46b3c291-be47-4a57-82c2-b82e84a9909f\") " pod="openshift-image-registry/node-ca-5nlhv" Jul 24 03:18:24.516615 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.516591 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d8wq\" (UniqueName: \"kubernetes.io/projected/1bc60e51-e1b5-4f68-a907-600eb7bc0e0b-kube-api-access-8d8wq\") pod \"network-metrics-daemon-gkmfm\" (UID: \"1bc60e51-e1b5-4f68-a907-600eb7bc0e0b\") " pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:18:24.545775 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:24.545742 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ae97c5152448a4aac7688f6e2b6cf38.slice/crio-c6d49c409dcbf9f6d3a4ab719d147ea84481e44bde98b5d27d094113f1940f05 WatchSource:0}: Error finding container c6d49c409dcbf9f6d3a4ab719d147ea84481e44bde98b5d27d094113f1940f05: Status 404 returned error can't find the container with id c6d49c409dcbf9f6d3a4ab719d147ea84481e44bde98b5d27d094113f1940f05 Jul 24 03:18:24.546244 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:24.546223 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0db6a30028026bdb73215df8b7b1a010.slice/crio-fabfb43c8879e7d6bd5c5d8f84f1d9a90ab57f8f4be88f7f7d5ebf0bc5b097f8 WatchSource:0}: Error finding container fabfb43c8879e7d6bd5c5d8f84f1d9a90ab57f8f4be88f7f7d5ebf0bc5b097f8: Status 404 returned error can't find the container with id fabfb43c8879e7d6bd5c5d8f84f1d9a90ab57f8f4be88f7f7d5ebf0bc5b097f8 Jul 24 03:18:24.550754 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.550739 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 03:18:24.620814 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.620724 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qmbvk" Jul 24 03:18:24.629468 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.629445 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-brtgx" Jul 24 03:18:24.631049 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:24.631024 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2c5797b_2cf9_4743_b334_7efa2192095a.slice/crio-d5208a50f9c8672cd9566d8e10334dc7e999b83810940008cc12c58932f39ce7 WatchSource:0}: Error finding container d5208a50f9c8672cd9566d8e10334dc7e999b83810940008cc12c58932f39ce7: Status 404 returned error can't find the container with id d5208a50f9c8672cd9566d8e10334dc7e999b83810940008cc12c58932f39ce7 Jul 24 03:18:24.637900 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:24.637875 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47ab9dde_f95f_4503_9087_b3ba282dc067.slice/crio-8a0b0dd03f5b3a68c1ff2d60f8d8edb29cb12a771b9ed8f8811fac069974062e WatchSource:0}: Error finding container 8a0b0dd03f5b3a68c1ff2d60f8d8edb29cb12a771b9ed8f8811fac069974062e: Status 404 returned error can't find the container with id 8a0b0dd03f5b3a68c1ff2d60f8d8edb29cb12a771b9ed8f8811fac069974062e Jul 24 03:18:24.646501 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.646483 2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 03:18:24.646587 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.646529 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-94swq" Jul 24 03:18:24.655769 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:24.655733 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4228fed_1593_4121_99cc_a5ccc6143da7.slice/crio-5da73bbc8facd58dda210cc14d64a88699c90f1b18411660647de085661353d2 WatchSource:0}: Error finding container 5da73bbc8facd58dda210cc14d64a88699c90f1b18411660647de085661353d2: Status 404 returned error can't find the container with id 5da73bbc8facd58dda210cc14d64a88699c90f1b18411660647de085661353d2 Jul 24 03:18:24.656936 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.656917 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jkzg8" Jul 24 03:18:24.665805 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:24.665781 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd59f65d_aa8f_4dfb_86fa_37c8523647ce.slice/crio-eea2a1b6ca99eafe9eadbf52aaf93ff550c8eb0ec6dd457dfb5cefa880f2de86 WatchSource:0}: Error finding container eea2a1b6ca99eafe9eadbf52aaf93ff550c8eb0ec6dd457dfb5cefa880f2de86: Status 404 returned error can't find the container with id eea2a1b6ca99eafe9eadbf52aaf93ff550c8eb0ec6dd457dfb5cefa880f2de86 Jul 24 03:18:24.675404 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.675386 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8pzd4" Jul 24 03:18:24.683081 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:24.683061 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85c2fe64_4089_4a8e_b108_74936e8093ec.slice/crio-5ea710074e37bc4c60a6275edf3506f91a2302c2b71b220ab74d6bf7ad64c7b8 WatchSource:0}: Error finding container 5ea710074e37bc4c60a6275edf3506f91a2302c2b71b220ab74d6bf7ad64c7b8: Status 404 returned error can't find the container with id 5ea710074e37bc4c60a6275edf3506f91a2302c2b71b220ab74d6bf7ad64c7b8 Jul 24 03:18:24.692111 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.692093 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6fg2j" Jul 24 03:18:24.700389 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:24.700369 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba246244_7a40_425a_a886_a732efbe7c24.slice/crio-ea5cef400fa69585b0b5223f04687140f0258a2a50091d4c50c5bcf0980798a8 WatchSource:0}: Error finding container ea5cef400fa69585b0b5223f04687140f0258a2a50091d4c50c5bcf0980798a8: Status 404 returned error can't find the container with id ea5cef400fa69585b0b5223f04687140f0258a2a50091d4c50c5bcf0980798a8 Jul 24 03:18:24.707929 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.707893 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gfdrz" Jul 24 03:18:24.715994 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.715965 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5nlhv" Jul 24 03:18:24.716265 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:24.716247 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fc751d2_6a75_413f_881c_5a3c92935de3.slice/crio-47b770e451a2f7656ed5c512ad5e237c882f590dec8902d884413a6953259bce WatchSource:0}: Error finding container 47b770e451a2f7656ed5c512ad5e237c882f590dec8902d884413a6953259bce: Status 404 returned error can't find the container with id 47b770e451a2f7656ed5c512ad5e237c882f590dec8902d884413a6953259bce Jul 24 03:18:24.720787 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:24.720765 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:24.725035 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:24.725012 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46b3c291_be47_4a57_82c2_b82e84a9909f.slice/crio-92b4f9988ee14617b34263a3699327cb01f700ed8e61b2c42ede67ff3d6e5271 WatchSource:0}: Error finding container 92b4f9988ee14617b34263a3699327cb01f700ed8e61b2c42ede67ff3d6e5271: Status 404 returned error can't find the container with id 92b4f9988ee14617b34263a3699327cb01f700ed8e61b2c42ede67ff3d6e5271 Jul 24 03:18:24.729789 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:18:24.729768 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b2e05ef_119c_4450_9827_f806c437f29f.slice/crio-31eb8e131732bfbafac6161935af53ab18bab7bfc37640a93b806e825fc756d7 WatchSource:0}: Error finding container 31eb8e131732bfbafac6161935af53ab18bab7bfc37640a93b806e825fc756d7: Status 404 returned error can't find the container with id 31eb8e131732bfbafac6161935af53ab18bab7bfc37640a93b806e825fc756d7 Jul 24 03:18:24.731688 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:24.731667 2579 kuberuntime_manager.go:1459] "Unhandled Error" err="container ovn-controller start failed in pod ovnkube-node-ffrlv_openshift-ovn-kubernetes(6b2e05ef-119c-4450-9827-f806c437f29f): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Jul 24 03:18:24.733186 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:24.733169 2579 kuberuntime_manager.go:1459] "Unhandled Error" err="container kube-rbac-proxy-node start failed in pod ovnkube-node-ffrlv_openshift-ovn-kubernetes(6b2e05ef-119c-4450-9827-f806c437f29f): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Jul 24 03:18:24.736419 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:24.736378 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"ovn-controller\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"ovn-acl-logging\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-node\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-ovn-metrics\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"northd\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"nbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"sbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"ovnkube-controller\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\": ErrImagePull: pull QPS exceeded\"]" pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" podUID="6b2e05ef-119c-4450-9827-f806c437f29f" Jul 24 03:18:25.006201 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:25.006109 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bc60e51-e1b5-4f68-a907-600eb7bc0e0b-metrics-certs\") pod \"network-metrics-daemon-gkmfm\" (UID: \"1bc60e51-e1b5-4f68-a907-600eb7bc0e0b\") " pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:18:25.006201 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:25.006149 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/041beb02-32e9-4b8e-8f9d-34ee8d1a72c6-original-pull-secret\") pod \"global-pull-secret-syncer-8hlhr\" (UID: \"041beb02-32e9-4b8e-8f9d-34ee8d1a72c6\") " pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:18:25.006392 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:25.006272 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 03:18:25.006392 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:25.006283 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 03:18:25.006392 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:25.006336 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/041beb02-32e9-4b8e-8f9d-34ee8d1a72c6-original-pull-secret podName:041beb02-32e9-4b8e-8f9d-34ee8d1a72c6 nodeName:}" failed. No retries permitted until 2026-07-24 03:18:26.006320406 +0000 UTC m=+3.281891736 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/041beb02-32e9-4b8e-8f9d-34ee8d1a72c6-original-pull-secret") pod "global-pull-secret-syncer-8hlhr" (UID: "041beb02-32e9-4b8e-8f9d-34ee8d1a72c6") : object "kube-system"/"original-pull-secret" not registered Jul 24 03:18:25.006392 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:25.006355 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bc60e51-e1b5-4f68-a907-600eb7bc0e0b-metrics-certs podName:1bc60e51-e1b5-4f68-a907-600eb7bc0e0b nodeName:}" failed. No retries permitted until 2026-07-24 03:18:26.006345373 +0000 UTC m=+3.281916702 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bc60e51-e1b5-4f68-a907-600eb7bc0e0b-metrics-certs") pod "network-metrics-daemon-gkmfm" (UID: "1bc60e51-e1b5-4f68-a907-600eb7bc0e0b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 03:18:25.107142 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:25.107103 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfqtr\" (UniqueName: \"kubernetes.io/projected/1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85-kube-api-access-qfqtr\") pod \"network-check-target-52h8l\" (UID: \"1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85\") " pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:18:25.107414 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:25.107392 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 03:18:25.107493 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:25.107421 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 03:18:25.107493 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:25.107437 2579 projected.go:196] Error preparing data for projected volume kube-api-access-qfqtr for pod openshift-network-diagnostics/network-check-target-52h8l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 03:18:25.107620 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:25.107507 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85-kube-api-access-qfqtr podName:1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85 nodeName:}" failed. No retries permitted until 2026-07-24 03:18:26.107488063 +0000 UTC m=+3.383059392 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qfqtr" (UniqueName: "kubernetes.io/projected/1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85-kube-api-access-qfqtr") pod "network-check-target-52h8l" (UID: "1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 03:18:25.353009 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:25.352904 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 03:13:25 +0000 UTC" deadline="2028-03-27 18:00:28.964838709 +0000 UTC" Jul 24 03:18:25.353009 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:25.352944 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14702h42m3.611900156s" Jul 24 03:18:25.451762 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:25.448973 2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 03:18:25.516967 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:25.516932 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:18:25.517155 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:25.517090 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkmfm" podUID="1bc60e51-e1b5-4f68-a907-600eb7bc0e0b" Jul 24 03:18:25.519855 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:25.517475 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:18:25.519855 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:25.517583 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hlhr" podUID="041beb02-32e9-4b8e-8f9d-34ee8d1a72c6" Jul 24 03:18:25.533567 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:25.533510 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" event={"ID":"6b2e05ef-119c-4450-9827-f806c437f29f","Type":"ContainerStarted","Data":"31eb8e131732bfbafac6161935af53ab18bab7bfc37640a93b806e825fc756d7"} Jul 24 03:18:25.545760 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:25.545712 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5nlhv" event={"ID":"46b3c291-be47-4a57-82c2-b82e84a9909f","Type":"ContainerStarted","Data":"92b4f9988ee14617b34263a3699327cb01f700ed8e61b2c42ede67ff3d6e5271"} Jul 24 03:18:25.568388 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:25.568352 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gfdrz" event={"ID":"5fc751d2-6a75-413f-881c-5a3c92935de3","Type":"ContainerStarted","Data":"47b770e451a2f7656ed5c512ad5e237c882f590dec8902d884413a6953259bce"} Jul 24 03:18:25.575867 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:25.575797 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"ovn-controller\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"ovn-acl-logging\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-node\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-ovn-metrics\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"northd\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"nbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"sbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"ovnkube-controller\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\": ErrImagePull: pull QPS exceeded\"]" pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" podUID="6b2e05ef-119c-4450-9827-f806c437f29f" Jul 24 03:18:25.576486 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:25.576429 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jkzg8" event={"ID":"fd59f65d-aa8f-4dfb-86fa-37c8523647ce","Type":"ContainerStarted","Data":"eea2a1b6ca99eafe9eadbf52aaf93ff550c8eb0ec6dd457dfb5cefa880f2de86"} Jul 24 03:18:25.583195 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:25.583169 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-94swq" event={"ID":"c4228fed-1593-4121-99cc-a5ccc6143da7","Type":"ContainerStarted","Data":"5da73bbc8facd58dda210cc14d64a88699c90f1b18411660647de085661353d2"} Jul 24 03:18:25.610550 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:25.610414 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qmbvk" event={"ID":"c2c5797b-2cf9-4743-b334-7efa2192095a","Type":"ContainerStarted","Data":"d5208a50f9c8672cd9566d8e10334dc7e999b83810940008cc12c58932f39ce7"} Jul 24 03:18:25.619639 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:25.619603 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-134.ec2.internal" event={"ID":"0db6a30028026bdb73215df8b7b1a010","Type":"ContainerStarted","Data":"fabfb43c8879e7d6bd5c5d8f84f1d9a90ab57f8f4be88f7f7d5ebf0bc5b097f8"} Jul 24 03:18:25.634522 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:25.634440 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6fg2j" event={"ID":"ba246244-7a40-425a-a886-a732efbe7c24","Type":"ContainerStarted","Data":"ea5cef400fa69585b0b5223f04687140f0258a2a50091d4c50c5bcf0980798a8"} Jul 24 03:18:25.639561 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:25.639530 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8pzd4" event={"ID":"85c2fe64-4089-4a8e-b108-74936e8093ec","Type":"ContainerStarted","Data":"5ea710074e37bc4c60a6275edf3506f91a2302c2b71b220ab74d6bf7ad64c7b8"} Jul 24 03:18:25.650426 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:25.650393 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-brtgx" event={"ID":"47ab9dde-f95f-4503-9087-b3ba282dc067","Type":"ContainerStarted","Data":"8a0b0dd03f5b3a68c1ff2d60f8d8edb29cb12a771b9ed8f8811fac069974062e"} Jul 24 03:18:25.676357 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:25.676309 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-134.ec2.internal" event={"ID":"0ae97c5152448a4aac7688f6e2b6cf38","Type":"ContainerStarted","Data":"c6d49c409dcbf9f6d3a4ab719d147ea84481e44bde98b5d27d094113f1940f05"} Jul 24 03:18:26.015703 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:26.015592 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bc60e51-e1b5-4f68-a907-600eb7bc0e0b-metrics-certs\") pod \"network-metrics-daemon-gkmfm\" (UID: \"1bc60e51-e1b5-4f68-a907-600eb7bc0e0b\") " pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:18:26.015703 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:26.015656 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/041beb02-32e9-4b8e-8f9d-34ee8d1a72c6-original-pull-secret\") pod \"global-pull-secret-syncer-8hlhr\" (UID: \"041beb02-32e9-4b8e-8f9d-34ee8d1a72c6\") " pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:18:26.015959 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:26.015841 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 03:18:26.015959 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:26.015913 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/041beb02-32e9-4b8e-8f9d-34ee8d1a72c6-original-pull-secret podName:041beb02-32e9-4b8e-8f9d-34ee8d1a72c6 nodeName:}" failed. No retries permitted until 2026-07-24 03:18:28.015892036 +0000 UTC m=+5.291463367 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/041beb02-32e9-4b8e-8f9d-34ee8d1a72c6-original-pull-secret") pod "global-pull-secret-syncer-8hlhr" (UID: "041beb02-32e9-4b8e-8f9d-34ee8d1a72c6") : object "kube-system"/"original-pull-secret" not registered Jul 24 03:18:26.016345 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:26.016324 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 03:18:26.016452 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:26.016378 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bc60e51-e1b5-4f68-a907-600eb7bc0e0b-metrics-certs podName:1bc60e51-e1b5-4f68-a907-600eb7bc0e0b nodeName:}" failed. No retries permitted until 2026-07-24 03:18:28.016362638 +0000 UTC m=+5.291933967 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bc60e51-e1b5-4f68-a907-600eb7bc0e0b-metrics-certs") pod "network-metrics-daemon-gkmfm" (UID: "1bc60e51-e1b5-4f68-a907-600eb7bc0e0b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 03:18:26.116890 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:26.116854 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfqtr\" (UniqueName: \"kubernetes.io/projected/1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85-kube-api-access-qfqtr\") pod \"network-check-target-52h8l\" (UID: \"1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85\") " pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:18:26.117090 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:26.117033 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 03:18:26.117090 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:26.117055 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 03:18:26.117090 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:26.117069 2579 projected.go:196] Error preparing data for projected volume kube-api-access-qfqtr for pod openshift-network-diagnostics/network-check-target-52h8l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 03:18:26.117270 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:26.117129 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85-kube-api-access-qfqtr podName:1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85 nodeName:}" failed. No retries permitted until 2026-07-24 03:18:28.117109062 +0000 UTC m=+5.392680399 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qfqtr" (UniqueName: "kubernetes.io/projected/1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85-kube-api-access-qfqtr") pod "network-check-target-52h8l" (UID: "1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 03:18:26.354164 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:26.354032 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 03:13:25 +0000 UTC" deadline="2027-12-28 22:56:11.803616349 +0000 UTC" Jul 24 03:18:26.354164 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:26.354100 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12547h37m45.449521259s" Jul 24 03:18:26.516524 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:26.515997 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:18:26.516524 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:26.516131 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52h8l" podUID="1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85" Jul 24 03:18:26.983063 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:26.983032 2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 03:18:27.322982 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:27.322897 2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 03:18:27.516140 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:27.516104 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:18:27.516630 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:27.516258 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkmfm" podUID="1bc60e51-e1b5-4f68-a907-600eb7bc0e0b" Jul 24 03:18:27.516630 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:27.516360 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:18:27.516630 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:27.516488 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hlhr" podUID="041beb02-32e9-4b8e-8f9d-34ee8d1a72c6" Jul 24 03:18:28.035482 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:28.035437 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bc60e51-e1b5-4f68-a907-600eb7bc0e0b-metrics-certs\") pod \"network-metrics-daemon-gkmfm\" (UID: \"1bc60e51-e1b5-4f68-a907-600eb7bc0e0b\") " pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:18:28.035702 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:28.035496 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/041beb02-32e9-4b8e-8f9d-34ee8d1a72c6-original-pull-secret\") pod \"global-pull-secret-syncer-8hlhr\" (UID: \"041beb02-32e9-4b8e-8f9d-34ee8d1a72c6\") " pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:18:28.035702 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:28.035612 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 03:18:28.035702 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:28.035613 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 03:18:28.035702 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:28.035676 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/041beb02-32e9-4b8e-8f9d-34ee8d1a72c6-original-pull-secret podName:041beb02-32e9-4b8e-8f9d-34ee8d1a72c6 nodeName:}" failed. No retries permitted until 2026-07-24 03:18:32.035658966 +0000 UTC m=+9.311230299 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/041beb02-32e9-4b8e-8f9d-34ee8d1a72c6-original-pull-secret") pod "global-pull-secret-syncer-8hlhr" (UID: "041beb02-32e9-4b8e-8f9d-34ee8d1a72c6") : object "kube-system"/"original-pull-secret" not registered Jul 24 03:18:28.035974 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:28.035697 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bc60e51-e1b5-4f68-a907-600eb7bc0e0b-metrics-certs podName:1bc60e51-e1b5-4f68-a907-600eb7bc0e0b nodeName:}" failed. No retries permitted until 2026-07-24 03:18:32.035686254 +0000 UTC m=+9.311257587 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bc60e51-e1b5-4f68-a907-600eb7bc0e0b-metrics-certs") pod "network-metrics-daemon-gkmfm" (UID: "1bc60e51-e1b5-4f68-a907-600eb7bc0e0b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 03:18:28.136289 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:28.136250 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfqtr\" (UniqueName: \"kubernetes.io/projected/1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85-kube-api-access-qfqtr\") pod \"network-check-target-52h8l\" (UID: \"1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85\") " pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:18:28.136497 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:28.136431 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 03:18:28.136581 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:28.136532 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 03:18:28.136581 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:28.136548 2579 projected.go:196] Error preparing data for projected volume kube-api-access-qfqtr for pod openshift-network-diagnostics/network-check-target-52h8l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 03:18:28.136711 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:28.136621 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85-kube-api-access-qfqtr podName:1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85 nodeName:}" failed. No retries permitted until 2026-07-24 03:18:32.13660181 +0000 UTC m=+9.412173146 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qfqtr" (UniqueName: "kubernetes.io/projected/1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85-kube-api-access-qfqtr") pod "network-check-target-52h8l" (UID: "1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 03:18:28.516093 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:28.515994 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:18:28.516277 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:28.516143 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52h8l" podUID="1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85" Jul 24 03:18:29.516430 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:29.516377 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:18:29.516947 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:29.516388 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:18:29.516947 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:29.516541 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hlhr" podUID="041beb02-32e9-4b8e-8f9d-34ee8d1a72c6" Jul 24 03:18:29.516947 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:29.516603 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkmfm" podUID="1bc60e51-e1b5-4f68-a907-600eb7bc0e0b" Jul 24 03:18:30.515841 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:30.515805 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:18:30.516017 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:30.515926 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52h8l" podUID="1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85" Jul 24 03:18:31.515816 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:31.515726 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:18:31.516275 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:31.515875 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkmfm" podUID="1bc60e51-e1b5-4f68-a907-600eb7bc0e0b" Jul 24 03:18:31.516275 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:31.515906 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:18:31.516275 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:31.516023 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hlhr" podUID="041beb02-32e9-4b8e-8f9d-34ee8d1a72c6" Jul 24 03:18:32.070912 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:32.070864 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bc60e51-e1b5-4f68-a907-600eb7bc0e0b-metrics-certs\") pod \"network-metrics-daemon-gkmfm\" (UID: \"1bc60e51-e1b5-4f68-a907-600eb7bc0e0b\") " pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:18:32.071108 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:32.070929 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/041beb02-32e9-4b8e-8f9d-34ee8d1a72c6-original-pull-secret\") pod \"global-pull-secret-syncer-8hlhr\" (UID: \"041beb02-32e9-4b8e-8f9d-34ee8d1a72c6\") " pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:18:32.071108 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:32.071061 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 03:18:32.071235 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:32.071127 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/041beb02-32e9-4b8e-8f9d-34ee8d1a72c6-original-pull-secret podName:041beb02-32e9-4b8e-8f9d-34ee8d1a72c6 nodeName:}" failed. No retries permitted until 2026-07-24 03:18:40.071107718 +0000 UTC m=+17.346679053 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/041beb02-32e9-4b8e-8f9d-34ee8d1a72c6-original-pull-secret") pod "global-pull-secret-syncer-8hlhr" (UID: "041beb02-32e9-4b8e-8f9d-34ee8d1a72c6") : object "kube-system"/"original-pull-secret" not registered Jul 24 03:18:32.071476 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:32.071412 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 03:18:32.071575 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:32.071491 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bc60e51-e1b5-4f68-a907-600eb7bc0e0b-metrics-certs podName:1bc60e51-e1b5-4f68-a907-600eb7bc0e0b nodeName:}" failed. No retries permitted until 2026-07-24 03:18:40.071471785 +0000 UTC m=+17.347043116 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bc60e51-e1b5-4f68-a907-600eb7bc0e0b-metrics-certs") pod "network-metrics-daemon-gkmfm" (UID: "1bc60e51-e1b5-4f68-a907-600eb7bc0e0b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 03:18:32.171950 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:32.171915 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfqtr\" (UniqueName: \"kubernetes.io/projected/1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85-kube-api-access-qfqtr\") pod \"network-check-target-52h8l\" (UID: \"1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85\") " pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:18:32.172135 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:32.172078 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 03:18:32.172135 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:32.172100 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 03:18:32.172135 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:32.172112 2579 projected.go:196] Error preparing data for projected volume kube-api-access-qfqtr for pod openshift-network-diagnostics/network-check-target-52h8l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 03:18:32.172315 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:32.172170 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85-kube-api-access-qfqtr podName:1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85 nodeName:}" failed. No retries permitted until 2026-07-24 03:18:40.172150915 +0000 UTC m=+17.447722249 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qfqtr" (UniqueName: "kubernetes.io/projected/1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85-kube-api-access-qfqtr") pod "network-check-target-52h8l" (UID: "1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 03:18:32.515917 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:32.515835 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:18:32.516365 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:32.515960 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52h8l" podUID="1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85" Jul 24 03:18:33.519623 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:33.519590 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:18:33.519623 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:33.519617 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:18:33.520093 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:33.519755 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hlhr" podUID="041beb02-32e9-4b8e-8f9d-34ee8d1a72c6" Jul 24 03:18:33.520093 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:33.519866 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkmfm" podUID="1bc60e51-e1b5-4f68-a907-600eb7bc0e0b" Jul 24 03:18:34.515650 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:34.515616 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:18:34.515828 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:34.515740 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52h8l" podUID="1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85" Jul 24 03:18:35.516023 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:35.515821 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:18:35.516532 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:35.515821 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:18:35.516532 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:35.516104 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hlhr" podUID="041beb02-32e9-4b8e-8f9d-34ee8d1a72c6" Jul 24 03:18:35.516532 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:35.516219 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkmfm" podUID="1bc60e51-e1b5-4f68-a907-600eb7bc0e0b" Jul 24 03:18:36.516123 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:36.516091 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:18:36.516573 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:36.516213 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52h8l" podUID="1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85" Jul 24 03:18:37.516047 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:37.516015 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:18:37.516234 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:37.516133 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkmfm" podUID="1bc60e51-e1b5-4f68-a907-600eb7bc0e0b" Jul 24 03:18:37.516234 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:37.516190 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:18:37.516630 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:37.516292 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hlhr" podUID="041beb02-32e9-4b8e-8f9d-34ee8d1a72c6" Jul 24 03:18:38.515565 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:38.515542 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:18:38.515755 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:38.515697 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52h8l" podUID="1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85" Jul 24 03:18:38.715989 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:38.715638 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-brtgx" event={"ID":"47ab9dde-f95f-4503-9087-b3ba282dc067","Type":"ContainerStarted","Data":"1d3618cc3a279316ba0bcfee3eac52f967717868475ee124d19ad93c381f13dc"} Jul 24 03:18:38.721367 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:38.721311 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-134.ec2.internal" event={"ID":"0ae97c5152448a4aac7688f6e2b6cf38","Type":"ContainerStarted","Data":"173798e160fdde717867ba5978d584a770c7040859a163fa21aaa89adf9fa4da"} Jul 24 03:18:38.734125 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:38.734023 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-brtgx" podStartSLOduration=1.018836799 podStartE2EDuration="14.733994224s" totalImagesPullingTime="13.715157425s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 03:18:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:18:38.732939033 +0000 UTC m=+16.008510383" watchObservedRunningTime="2026-07-24 03:18:38.733994224 +0000 UTC m=+16.009565576" Jul 24 03:18:38.744609 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:38.744545 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-134.ec2.internal" podStartSLOduration=13.744533617 podStartE2EDuration="13.744533617s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 03:18:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:18:38.744462523 +0000 UTC m=+16.020033872" watchObservedRunningTime="2026-07-24 03:18:38.744533617 +0000 UTC m=+16.020104968" Jul 24 03:18:39.516456 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:39.516255 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:18:39.516620 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:39.516272 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:18:39.516620 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:39.516545 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hlhr" podUID="041beb02-32e9-4b8e-8f9d-34ee8d1a72c6" Jul 24 03:18:39.516754 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:39.516687 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkmfm" podUID="1bc60e51-e1b5-4f68-a907-600eb7bc0e0b" Jul 24 03:18:39.724215 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:39.724184 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5nlhv" event={"ID":"46b3c291-be47-4a57-82c2-b82e84a9909f","Type":"ContainerStarted","Data":"a28c4760bec499926cd6efb56609f5111ca0fde78b2e413775dd63a46ee7c721"} Jul 24 03:18:39.725460 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:39.725439 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gfdrz" event={"ID":"5fc751d2-6a75-413f-881c-5a3c92935de3","Type":"ContainerStarted","Data":"756667a76e7153f5a5c9ff7dacb5f5e0f1c3f741f18b7de1175ccb07baea6a55"} Jul 24 03:18:39.728400 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:39.728368 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jkzg8" event={"ID":"fd59f65d-aa8f-4dfb-86fa-37c8523647ce","Type":"ContainerStarted","Data":"4ccb04fff39f39ac8b373781c623ada5c8c7c3ac12302c5212abdc441c79ea2a"} Jul 24 03:18:39.730559 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:39.730533 2579 generic.go:356] "Generic (PLEG): container finished" podID="c4228fed-1593-4121-99cc-a5ccc6143da7" containerID="99b8416335066d4a317661fe21e2a0684249df66bc54632c0f03b4086c6507ef" exitCode=0 Jul 24 03:18:39.730656 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:39.730612 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-94swq" event={"ID":"c4228fed-1593-4121-99cc-a5ccc6143da7","Type":"ContainerDied","Data":"99b8416335066d4a317661fe21e2a0684249df66bc54632c0f03b4086c6507ef"} Jul 24 03:18:39.732073 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:39.732054 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qmbvk" event={"ID":"c2c5797b-2cf9-4743-b334-7efa2192095a","Type":"ContainerStarted","Data":"a9e13ffa1e9a7603ff2e9992d0e2070b0b10093fab5deeae2175a8e5f11e06e7"} Jul 24 03:18:39.733617 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:39.733593 2579 generic.go:356] "Generic (PLEG): container finished" podID="0db6a30028026bdb73215df8b7b1a010" containerID="229ad4b8d8f422dc69a98f58a5dba0bc30ad77a0c75c2d26b5e74a53ca6101ce" exitCode=0 Jul 24 03:18:39.733744 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:39.733698 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-134.ec2.internal" event={"ID":"0db6a30028026bdb73215df8b7b1a010","Type":"ContainerDied","Data":"229ad4b8d8f422dc69a98f58a5dba0bc30ad77a0c75c2d26b5e74a53ca6101ce"} Jul 24 03:18:39.735859 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:39.735472 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6fg2j" event={"ID":"ba246244-7a40-425a-a886-a732efbe7c24","Type":"ContainerStarted","Data":"0c62138a9069cda6da2c74c6c8eaeb9d186d4a60a547588db84da32d6e9e3294"} Jul 24 03:18:39.743240 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:39.743199 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5nlhv" podStartSLOduration=2.129340254 podStartE2EDuration="15.743182465s" totalImagesPullingTime="13.613842211s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 03:18:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:18:39.742757833 +0000 UTC m=+17.018329184" watchObservedRunningTime="2026-07-24 03:18:39.743182465 +0000 UTC m=+17.018753817" Jul 24 03:18:39.755922 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:39.755871 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qmbvk" podStartSLOduration=2.048070261 podStartE2EDuration="15.755859445s" totalImagesPullingTime="13.707789184s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:18:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:18:39.755424636 +0000 UTC m=+17.030995989" watchObservedRunningTime="2026-07-24 03:18:39.755859445 +0000 UTC m=+17.031430794" Jul 24 03:18:39.789091 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:39.789050 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-jkzg8" podStartSLOduration=1.832523722 podStartE2EDuration="15.78903549s" totalImagesPullingTime="13.956511768s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 03:18:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:18:39.788983764 +0000 UTC m=+17.064555124" watchObservedRunningTime="2026-07-24 03:18:39.78903549 +0000 UTC m=+17.064606840" Jul 24 03:18:39.801160 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:39.801107 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-gfdrz" podStartSLOduration=2.17829913 podStartE2EDuration="15.801090509s" totalImagesPullingTime="13.622791379s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 03:18:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:18:39.800976675 +0000 UTC m=+17.076548027" watchObservedRunningTime="2026-07-24 03:18:39.801090509 +0000 UTC m=+17.076661859" Jul 24 03:18:39.824041 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:39.824014 2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 24 03:18:40.132100 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:40.131963 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bc60e51-e1b5-4f68-a907-600eb7bc0e0b-metrics-certs\") pod \"network-metrics-daemon-gkmfm\" (UID: \"1bc60e51-e1b5-4f68-a907-600eb7bc0e0b\") " pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:18:40.132100 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:40.132068 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 03:18:40.132320 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:40.132108 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/041beb02-32e9-4b8e-8f9d-34ee8d1a72c6-original-pull-secret\") pod \"global-pull-secret-syncer-8hlhr\" (UID: \"041beb02-32e9-4b8e-8f9d-34ee8d1a72c6\") " pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:18:40.132320 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:40.132144 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bc60e51-e1b5-4f68-a907-600eb7bc0e0b-metrics-certs podName:1bc60e51-e1b5-4f68-a907-600eb7bc0e0b nodeName:}" failed. No retries permitted until 2026-07-24 03:18:56.132124156 +0000 UTC m=+33.407695504 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bc60e51-e1b5-4f68-a907-600eb7bc0e0b-metrics-certs") pod "network-metrics-daemon-gkmfm" (UID: "1bc60e51-e1b5-4f68-a907-600eb7bc0e0b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 03:18:40.132320 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:40.132207 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 03:18:40.132320 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:40.132256 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/041beb02-32e9-4b8e-8f9d-34ee8d1a72c6-original-pull-secret podName:041beb02-32e9-4b8e-8f9d-34ee8d1a72c6 nodeName:}" failed. No retries permitted until 2026-07-24 03:18:56.132242114 +0000 UTC m=+33.407813456 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/041beb02-32e9-4b8e-8f9d-34ee8d1a72c6-original-pull-secret") pod "global-pull-secret-syncer-8hlhr" (UID: "041beb02-32e9-4b8e-8f9d-34ee8d1a72c6") : object "kube-system"/"original-pull-secret" not registered Jul 24 03:18:40.233376 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:40.233335 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfqtr\" (UniqueName: \"kubernetes.io/projected/1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85-kube-api-access-qfqtr\") pod \"network-check-target-52h8l\" (UID: \"1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85\") " pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:18:40.233553 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:40.233535 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 03:18:40.233617 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:40.233564 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 03:18:40.233617 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:40.233578 2579 projected.go:196] Error preparing data for projected volume kube-api-access-qfqtr for pod openshift-network-diagnostics/network-check-target-52h8l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 03:18:40.233749 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:40.233646 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85-kube-api-access-qfqtr podName:1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85 nodeName:}" failed. No retries permitted until 2026-07-24 03:18:56.23362696 +0000 UTC m=+33.509198312 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qfqtr" (UniqueName: "kubernetes.io/projected/1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85-kube-api-access-qfqtr") pod "network-check-target-52h8l" (UID: "1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 03:18:40.354745 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:40.353920 2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-24T03:18:39.82403765Z","UUID":"e0dcac5d-67a0-4e4e-b1b8-c006b5d53f3f","Handler":null,"Name":"","Endpoint":""} Jul 24 03:18:40.356270 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:40.356015 2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 24 03:18:40.356270 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:40.356061 2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 24 03:18:40.515438 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:40.515358 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:18:40.515574 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:40.515491 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52h8l" podUID="1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85" Jul 24 03:18:40.742227 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:40.742176 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-134.ec2.internal" event={"ID":"0db6a30028026bdb73215df8b7b1a010","Type":"ContainerStarted","Data":"b3797bf39c254db919e5a71cd307be6d5a5e29ebc2dac3e01f2f738ac4c5e32f"} Jul 24 03:18:40.746141 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:40.746109 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6fg2j" event={"ID":"ba246244-7a40-425a-a886-a732efbe7c24","Type":"ContainerStarted","Data":"f00cf56291f2ea4d5114437fc64538e6e632ab32775983aa6d18551a38518dd6"} Jul 24 03:18:40.749773 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:40.749717 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8pzd4" event={"ID":"85c2fe64-4089-4a8e-b108-74936e8093ec","Type":"ContainerStarted","Data":"fafefaa367256a6ada5dae4039330d6a0a287f3a699246818a8a4c9738e0ea97"} Jul 24 03:18:40.756997 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:40.756385 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-134.ec2.internal" podStartSLOduration=15.756367761 podStartE2EDuration="15.756367761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 03:18:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:18:40.755737363 +0000 UTC m=+18.031308713" watchObservedRunningTime="2026-07-24 03:18:40.756367761 +0000 UTC m=+18.031939111" Jul 24 03:18:40.770094 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:40.769978 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8pzd4" podStartSLOduration=3.112472053 podStartE2EDuration="16.769958265s" totalImagesPullingTime="13.657486212s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 03:18:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:18:40.769869593 +0000 UTC m=+18.045440944" watchObservedRunningTime="2026-07-24 03:18:40.769958265 +0000 UTC m=+18.045529618" Jul 24 03:18:41.519708 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:41.519679 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:18:41.519879 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:41.519679 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:18:41.519936 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:41.519880 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkmfm" podUID="1bc60e51-e1b5-4f68-a907-600eb7bc0e0b" Jul 24 03:18:41.520018 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:41.519993 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hlhr" podUID="041beb02-32e9-4b8e-8f9d-34ee8d1a72c6" Jul 24 03:18:42.516161 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:42.516131 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:18:42.516923 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:42.516243 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52h8l" podUID="1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85" Jul 24 03:18:42.657386 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:42.657149 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qmbvk" Jul 24 03:18:42.657806 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:42.657781 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qmbvk" Jul 24 03:18:42.756473 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:42.756439 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6fg2j" event={"ID":"ba246244-7a40-425a-a886-a732efbe7c24","Type":"ContainerStarted","Data":"e18b262d2ebeba2a1e5381eea23e7f9e518098840e45abcfb3e6590dac7a58db"} Jul 24 03:18:42.756940 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:42.756881 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qmbvk" Jul 24 03:18:42.757240 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:42.757217 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qmbvk" Jul 24 03:18:42.779846 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:42.779734 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6fg2j" podStartSLOduration=2.383822803 podStartE2EDuration="18.77971672s" totalImagesPullingTime="16.395893917s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 03:18:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:18:42.779474598 +0000 UTC m=+20.055045948" watchObservedRunningTime="2026-07-24 03:18:42.77971672 +0000 UTC m=+20.055288072" Jul 24 03:18:43.464926 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:43.464894 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gfdrz_5fc751d2-6a75-413f-881c-5a3c92935de3/dns-node-resolver/0.log" Jul 24 03:18:43.517179 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:43.517149 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:18:43.517628 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:43.517268 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkmfm" podUID="1bc60e51-e1b5-4f68-a907-600eb7bc0e0b" Jul 24 03:18:43.517748 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:43.517729 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:18:43.517856 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:43.517830 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hlhr" podUID="041beb02-32e9-4b8e-8f9d-34ee8d1a72c6" Jul 24 03:18:44.515641 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:44.515611 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:18:44.515806 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:44.515778 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52h8l" podUID="1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85" Jul 24 03:18:44.646148 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:44.646127 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5nlhv_46b3c291-be47-4a57-82c2-b82e84a9909f/node-ca/0.log" Jul 24 03:18:44.761920 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:44.761840 2579 generic.go:356] "Generic (PLEG): container finished" podID="c4228fed-1593-4121-99cc-a5ccc6143da7" containerID="0e76c89a8a2027471161cc707726c3c8d09f6e4a65f30d4bc95c82b8f773b1fa" exitCode=0 Jul 24 03:18:44.761920 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:44.761896 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-94swq" event={"ID":"c4228fed-1593-4121-99cc-a5ccc6143da7","Type":"ContainerDied","Data":"0e76c89a8a2027471161cc707726c3c8d09f6e4a65f30d4bc95c82b8f773b1fa"} Jul 24 03:18:45.515997 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:45.515959 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:18:45.516150 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:45.516012 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:18:45.516150 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:45.516124 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkmfm" podUID="1bc60e51-e1b5-4f68-a907-600eb7bc0e0b" Jul 24 03:18:45.516296 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:45.516271 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hlhr" podUID="041beb02-32e9-4b8e-8f9d-34ee8d1a72c6" Jul 24 03:18:45.766736 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:45.766635 2579 generic.go:356] "Generic (PLEG): container finished" podID="c4228fed-1593-4121-99cc-a5ccc6143da7" containerID="cf504d75035c9d0382e1570a074e7b89c9af2ea0891238a2e948c575ec605459" exitCode=0 Jul 24 03:18:45.766736 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:45.766700 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-94swq" event={"ID":"c4228fed-1593-4121-99cc-a5ccc6143da7","Type":"ContainerDied","Data":"cf504d75035c9d0382e1570a074e7b89c9af2ea0891238a2e948c575ec605459"} Jul 24 03:18:46.515416 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:46.515391 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:18:46.515536 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:46.515510 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52h8l" podUID="1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85" Jul 24 03:18:46.771787 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:46.771698 2579 generic.go:356] "Generic (PLEG): container finished" podID="c4228fed-1593-4121-99cc-a5ccc6143da7" containerID="eeabbd360cdada7aa604be49d663fc1fb87d8ce6410199d62e6f2b8e10397695" exitCode=0 Jul 24 03:18:46.771787 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:46.771742 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-94swq" event={"ID":"c4228fed-1593-4121-99cc-a5ccc6143da7","Type":"ContainerDied","Data":"eeabbd360cdada7aa604be49d663fc1fb87d8ce6410199d62e6f2b8e10397695"} Jul 24 03:18:47.516055 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:47.516020 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:18:47.516218 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:47.516157 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkmfm" podUID="1bc60e51-e1b5-4f68-a907-600eb7bc0e0b" Jul 24 03:18:47.516218 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:47.516025 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:18:47.516341 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:47.516318 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hlhr" podUID="041beb02-32e9-4b8e-8f9d-34ee8d1a72c6" Jul 24 03:18:48.516134 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:48.516102 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:18:48.516549 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:48.516224 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52h8l" podUID="1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85" Jul 24 03:18:49.515267 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:49.515236 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:18:49.515267 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:49.515251 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:18:49.515508 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:49.515363 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hlhr" podUID="041beb02-32e9-4b8e-8f9d-34ee8d1a72c6" Jul 24 03:18:49.515565 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:49.515541 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkmfm" podUID="1bc60e51-e1b5-4f68-a907-600eb7bc0e0b" Jul 24 03:18:50.515330 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:50.515296 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:18:50.515815 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:50.515423 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52h8l" podUID="1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85" Jul 24 03:18:51.518901 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:51.518868 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:18:51.519301 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:51.518986 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hlhr" podUID="041beb02-32e9-4b8e-8f9d-34ee8d1a72c6" Jul 24 03:18:51.519301 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:51.519041 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:18:51.519301 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:51.519159 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkmfm" podUID="1bc60e51-e1b5-4f68-a907-600eb7bc0e0b" Jul 24 03:18:51.785102 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:51.784898 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ffrlv_6b2e05ef-119c-4450-9827-f806c437f29f/ovn-acl-logging/0.log" Jul 24 03:18:51.785456 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:51.785433 2579 generic.go:356] "Generic (PLEG): container finished" podID="6b2e05ef-119c-4450-9827-f806c437f29f" containerID="ad3b8bb8e378d5818bf5f565008cb9256ed445403ed44a31f8c953bcce5b4f14" exitCode=1 Jul 24 03:18:51.785553 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:51.785492 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" event={"ID":"6b2e05ef-119c-4450-9827-f806c437f29f","Type":"ContainerStarted","Data":"0b44b1f41217e3c20b26d53649b67ea8f0dbd0cc5c273e6df8fb2dfc851f2d71"} Jul 24 03:18:51.785614 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:51.785567 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" event={"ID":"6b2e05ef-119c-4450-9827-f806c437f29f","Type":"ContainerDied","Data":"ad3b8bb8e378d5818bf5f565008cb9256ed445403ed44a31f8c953bcce5b4f14"} Jul 24 03:18:51.785736 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:51.785618 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" event={"ID":"6b2e05ef-119c-4450-9827-f806c437f29f","Type":"ContainerStarted","Data":"e129cfedf6101a3adb8c7d05fc2efc28c140cf0569418b2ec77562bf386bc3f6"} Jul 24 03:18:52.515337 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:52.515301 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:18:52.515516 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:52.515436 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52h8l" podUID="1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85" Jul 24 03:18:52.790891 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:52.790821 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ffrlv_6b2e05ef-119c-4450-9827-f806c437f29f/ovn-acl-logging/0.log" Jul 24 03:18:52.791460 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:52.791176 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" event={"ID":"6b2e05ef-119c-4450-9827-f806c437f29f","Type":"ContainerStarted","Data":"00db1361fa3096a063015d223b2784ae405183d2828af54b6e389f1bbf2d39bb"} Jul 24 03:18:52.791460 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:52.791256 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" event={"ID":"6b2e05ef-119c-4450-9827-f806c437f29f","Type":"ContainerStarted","Data":"066bcc7b68d312144f883a4b78c196fccf28bf261432c7e6917dfdd4a7721bf6"} Jul 24 03:18:52.791460 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:52.791306 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" event={"ID":"6b2e05ef-119c-4450-9827-f806c437f29f","Type":"ContainerStarted","Data":"585b1ffcae713fb9f87c8df2dada0d1a02098e5707040dcb793f69b5c8597cc8"} Jul 24 03:18:53.516055 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:53.516019 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:18:53.516276 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:53.516128 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hlhr" podUID="041beb02-32e9-4b8e-8f9d-34ee8d1a72c6" Jul 24 03:18:53.516276 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:53.516189 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:18:53.516402 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:53.516307 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkmfm" podUID="1bc60e51-e1b5-4f68-a907-600eb7bc0e0b" Jul 24 03:18:54.515938 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:54.515848 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:18:54.516483 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:54.515983 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52h8l" podUID="1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85" Jul 24 03:18:55.516037 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:55.515968 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:18:55.516417 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:55.516070 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hlhr" podUID="041beb02-32e9-4b8e-8f9d-34ee8d1a72c6" Jul 24 03:18:55.516417 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:55.516130 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:18:55.516417 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:55.516192 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkmfm" podUID="1bc60e51-e1b5-4f68-a907-600eb7bc0e0b" Jul 24 03:18:55.799228 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:55.799162 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ffrlv_6b2e05ef-119c-4450-9827-f806c437f29f/ovn-acl-logging/0.log" Jul 24 03:18:55.799585 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:55.799563 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" event={"ID":"6b2e05ef-119c-4450-9827-f806c437f29f","Type":"ContainerStarted","Data":"dfb7a005f688b48e8d65ac6fb11e8596696936f4ef3b2ee96b896f1a8517f248"} Jul 24 03:18:55.802151 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:55.802125 2579 generic.go:356] "Generic (PLEG): container finished" podID="c4228fed-1593-4121-99cc-a5ccc6143da7" containerID="1ca22dfe8686a8739777a174cb75d52ce38acb81702902716bccad968068926d" exitCode=0 Jul 24 03:18:55.802252 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:55.802171 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-94swq" event={"ID":"c4228fed-1593-4121-99cc-a5ccc6143da7","Type":"ContainerDied","Data":"1ca22dfe8686a8739777a174cb75d52ce38acb81702902716bccad968068926d"} Jul 24 03:18:56.158348 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:56.158312 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bc60e51-e1b5-4f68-a907-600eb7bc0e0b-metrics-certs\") pod \"network-metrics-daemon-gkmfm\" (UID: \"1bc60e51-e1b5-4f68-a907-600eb7bc0e0b\") " pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:18:56.158348 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:56.158350 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/041beb02-32e9-4b8e-8f9d-34ee8d1a72c6-original-pull-secret\") pod \"global-pull-secret-syncer-8hlhr\" (UID: \"041beb02-32e9-4b8e-8f9d-34ee8d1a72c6\") " pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:18:56.158540 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:56.158453 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 03:18:56.158540 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:56.158456 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 03:18:56.158540 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:56.158501 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/041beb02-32e9-4b8e-8f9d-34ee8d1a72c6-original-pull-secret podName:041beb02-32e9-4b8e-8f9d-34ee8d1a72c6 nodeName:}" failed. No retries permitted until 2026-07-24 03:19:28.158489217 +0000 UTC m=+65.434060545 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/041beb02-32e9-4b8e-8f9d-34ee8d1a72c6-original-pull-secret") pod "global-pull-secret-syncer-8hlhr" (UID: "041beb02-32e9-4b8e-8f9d-34ee8d1a72c6") : object "kube-system"/"original-pull-secret" not registered Jul 24 03:18:56.158540 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:56.158515 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bc60e51-e1b5-4f68-a907-600eb7bc0e0b-metrics-certs podName:1bc60e51-e1b5-4f68-a907-600eb7bc0e0b nodeName:}" failed. No retries permitted until 2026-07-24 03:19:28.158508677 +0000 UTC m=+65.434080005 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bc60e51-e1b5-4f68-a907-600eb7bc0e0b-metrics-certs") pod "network-metrics-daemon-gkmfm" (UID: "1bc60e51-e1b5-4f68-a907-600eb7bc0e0b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 03:18:56.258815 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:56.258781 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfqtr\" (UniqueName: \"kubernetes.io/projected/1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85-kube-api-access-qfqtr\") pod \"network-check-target-52h8l\" (UID: \"1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85\") " pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:18:56.258950 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:56.258916 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 03:18:56.258950 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:56.258931 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 03:18:56.258950 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:56.258940 2579 projected.go:196] Error preparing data for projected volume kube-api-access-qfqtr for pod openshift-network-diagnostics/network-check-target-52h8l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 03:18:56.259043 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:56.258988 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85-kube-api-access-qfqtr podName:1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85 nodeName:}" failed. No retries permitted until 2026-07-24 03:19:28.25897194 +0000 UTC m=+65.534543284 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qfqtr" (UniqueName: "kubernetes.io/projected/1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85-kube-api-access-qfqtr") pod "network-check-target-52h8l" (UID: "1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 03:18:56.515826 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:56.515761 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:18:56.515958 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:56.515856 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52h8l" podUID="1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85" Jul 24 03:18:56.807016 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:56.806937 2579 generic.go:356] "Generic (PLEG): container finished" podID="c4228fed-1593-4121-99cc-a5ccc6143da7" containerID="ec72bb3a049813612bbf5ff6acb2a7b330de7cf02ba9c9f3910c693933590933" exitCode=0 Jul 24 03:18:56.807016 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:56.806989 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-94swq" event={"ID":"c4228fed-1593-4121-99cc-a5ccc6143da7","Type":"ContainerDied","Data":"ec72bb3a049813612bbf5ff6acb2a7b330de7cf02ba9c9f3910c693933590933"} Jul 24 03:18:57.515702 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:57.515507 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:18:57.516010 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:57.515573 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:18:57.516010 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:57.515788 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkmfm" podUID="1bc60e51-e1b5-4f68-a907-600eb7bc0e0b" Jul 24 03:18:57.516010 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:57.515879 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hlhr" podUID="041beb02-32e9-4b8e-8f9d-34ee8d1a72c6" Jul 24 03:18:57.812985 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:57.812899 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-94swq" event={"ID":"c4228fed-1593-4121-99cc-a5ccc6143da7","Type":"ContainerStarted","Data":"4d3077756343e4500c0affadd333045b7108b531896b82410c43f4b267a683e7"} Jul 24 03:18:57.816010 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:57.815990 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ffrlv_6b2e05ef-119c-4450-9827-f806c437f29f/ovn-acl-logging/0.log" Jul 24 03:18:57.816304 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:57.816279 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" event={"ID":"6b2e05ef-119c-4450-9827-f806c437f29f","Type":"ContainerStarted","Data":"329e196711a4135f75ebd21db26e324eb7783d2b1876da3e6b11e677b98a5709"} Jul 24 03:18:57.816475 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:57.816458 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:57.816548 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:57.816523 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:57.816730 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:57.816705 2579 scope.go:117] "RemoveContainer" containerID="ad3b8bb8e378d5818bf5f565008cb9256ed445403ed44a31f8c953bcce5b4f14" Jul 24 03:18:57.835054 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:57.835034 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:57.835302 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:57.835266 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-94swq" podStartSLOduration=5.7059544110000004 podStartE2EDuration="33.835254076s" totalImagesPullingTime="28.129299665s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 03:18:24 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:18:57.832478137 +0000 UTC m=+35.108049490" watchObservedRunningTime="2026-07-24 03:18:57.835254076 +0000 UTC m=+35.110825425" Jul 24 03:18:58.515284 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:58.515254 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:18:58.515499 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:58.515364 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52h8l" podUID="1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85" Jul 24 03:18:58.821839 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:58.821757 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ffrlv_6b2e05ef-119c-4450-9827-f806c437f29f/ovn-acl-logging/0.log" Jul 24 03:18:58.822253 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:58.822168 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" event={"ID":"6b2e05ef-119c-4450-9827-f806c437f29f","Type":"ContainerStarted","Data":"bd4e67d3c5560aa64bf9bb124e20149a9bfdcc5ee404eabac03c0fb7882df0e4"} Jul 24 03:18:58.822615 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:58.822590 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:58.837349 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:58.837331 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:18:58.846066 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:58.846029 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" podStartSLOduration=8.193302723 podStartE2EDuration="34.846015348s" totalImagesPullingTime="26.652712625s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 03:18:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=2 observedRunningTime="2026-07-24 03:18:58.845161618 +0000 UTC m=+36.120732969" watchObservedRunningTime="2026-07-24 03:18:58.846015348 +0000 UTC m=+36.121586699" Jul 24 03:18:59.515986 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:59.515952 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:18:59.516177 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:59.516066 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hlhr" podUID="041beb02-32e9-4b8e-8f9d-34ee8d1a72c6" Jul 24 03:18:59.516177 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:18:59.516120 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:18:59.516303 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:18:59.516209 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkmfm" podUID="1bc60e51-e1b5-4f68-a907-600eb7bc0e0b" Jul 24 03:19:00.515634 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:00.515606 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:19:00.516008 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:19:00.515712 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52h8l" podUID="1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85" Jul 24 03:19:01.515990 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:01.515958 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:19:01.516334 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:19:01.516098 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkmfm" podUID="1bc60e51-e1b5-4f68-a907-600eb7bc0e0b" Jul 24 03:19:01.516334 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:01.516143 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:19:01.516334 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:19:01.516224 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hlhr" podUID="041beb02-32e9-4b8e-8f9d-34ee8d1a72c6" Jul 24 03:19:02.515800 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:02.515772 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:19:02.515971 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:19:02.515863 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52h8l" podUID="1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85" Jul 24 03:19:03.516340 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:03.516305 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:19:03.516742 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:19:03.516386 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hlhr" podUID="041beb02-32e9-4b8e-8f9d-34ee8d1a72c6" Jul 24 03:19:03.516742 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:03.516487 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:19:03.516742 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:19:03.516604 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkmfm" podUID="1bc60e51-e1b5-4f68-a907-600eb7bc0e0b" Jul 24 03:19:04.516123 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:04.516097 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:19:04.516277 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:19:04.516180 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52h8l" podUID="1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85" Jul 24 03:19:05.515925 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:05.515897 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:19:05.516283 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:19:05.515990 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hlhr" podUID="041beb02-32e9-4b8e-8f9d-34ee8d1a72c6" Jul 24 03:19:05.516283 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:05.516127 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:19:05.516283 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:19:05.516181 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkmfm" podUID="1bc60e51-e1b5-4f68-a907-600eb7bc0e0b" Jul 24 03:19:06.515748 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:06.515718 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:19:06.515918 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:19:06.515813 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52h8l" podUID="1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85" Jul 24 03:19:07.515913 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:07.515882 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:19:07.516290 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:19:07.516021 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkmfm" podUID="1bc60e51-e1b5-4f68-a907-600eb7bc0e0b" Jul 24 03:19:07.516290 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:07.516066 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:19:07.516290 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:19:07.516154 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hlhr" podUID="041beb02-32e9-4b8e-8f9d-34ee8d1a72c6" Jul 24 03:19:08.123773 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:08.123525 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-52h8l"] Jul 24 03:19:08.123932 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:08.123853 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:19:08.123986 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:19:08.123951 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52h8l" podUID="1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85" Jul 24 03:19:08.126683 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:08.126555 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8hlhr"] Jul 24 03:19:08.126812 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:08.126694 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:19:08.126812 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:19:08.126801 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hlhr" podUID="041beb02-32e9-4b8e-8f9d-34ee8d1a72c6" Jul 24 03:19:08.127483 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:08.127456 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gkmfm"] Jul 24 03:19:08.127584 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:08.127552 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:19:08.127677 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:19:08.127643 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkmfm" podUID="1bc60e51-e1b5-4f68-a907-600eb7bc0e0b" Jul 24 03:19:09.516012 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:09.515934 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:19:09.516510 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:09.515934 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:19:09.516510 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:19:09.516053 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hlhr" podUID="041beb02-32e9-4b8e-8f9d-34ee8d1a72c6" Jul 24 03:19:09.516510 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:09.515948 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:19:09.516510 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:19:09.516108 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkmfm" podUID="1bc60e51-e1b5-4f68-a907-600eb7bc0e0b" Jul 24 03:19:09.516510 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:19:09.516181 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52h8l" podUID="1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85" Jul 24 03:19:11.515755 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:11.515724 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:19:11.515755 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:11.515743 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:19:11.515755 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:11.515724 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:19:11.516255 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:19:11.515837 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hlhr" podUID="041beb02-32e9-4b8e-8f9d-34ee8d1a72c6" Jul 24 03:19:11.516255 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:19:11.515870 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52h8l" podUID="1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85" Jul 24 03:19:11.516255 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:19:11.515950 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkmfm" podUID="1bc60e51-e1b5-4f68-a907-600eb7bc0e0b" Jul 24 03:19:12.522546 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.522473 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-134.ec2.internal" event="NodeReady" Jul 24 03:19:12.522953 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.522592 2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 24 03:19:12.573660 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.573633 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-ts9wj"] Jul 24 03:19:12.575399 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.575381 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-ts9wj" Jul 24 03:19:12.576598 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.576577 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-rbhtr"] Jul 24 03:19:12.577827 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.577808 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 24 03:19:12.577970 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.577948 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-h286f\"" Jul 24 03:19:12.578929 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.578911 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rbhtr" Jul 24 03:19:12.581508 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.581484 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 24 03:19:12.581939 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.581921 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5zwrl"] Jul 24 03:19:12.583410 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.582988 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gd8s9\"" Jul 24 03:19:12.583410 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.583237 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 24 03:19:12.583787 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.583767 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5zwrl" Jul 24 03:19:12.585550 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.585519 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b4a79ed4-fe4a-407b-b0de-88c3206cd524-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-ts9wj\" (UID: \"b4a79ed4-fe4a-407b-b0de-88c3206cd524\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-ts9wj" Jul 24 03:19:12.585804 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.585776 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/642b4433-a01d-40ce-a14c-7dd42ab4e837-tmp-dir\") pod \"dns-default-rbhtr\" (UID: \"642b4433-a01d-40ce-a14c-7dd42ab4e837\") " pod="openshift-dns/dns-default-rbhtr" Jul 24 03:19:12.586076 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.586044 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/642b4433-a01d-40ce-a14c-7dd42ab4e837-config-volume\") pod \"dns-default-rbhtr\" (UID: \"642b4433-a01d-40ce-a14c-7dd42ab4e837\") " pod="openshift-dns/dns-default-rbhtr" Jul 24 03:19:12.586283 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.586265 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8l59q\" (UniqueName: \"kubernetes.io/projected/642b4433-a01d-40ce-a14c-7dd42ab4e837-kube-api-access-8l59q\") pod \"dns-default-rbhtr\" (UID: \"642b4433-a01d-40ce-a14c-7dd42ab4e837\") " pod="openshift-dns/dns-default-rbhtr" Jul 24 03:19:12.586407 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.586392 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/642b4433-a01d-40ce-a14c-7dd42ab4e837-metrics-tls\") pod \"dns-default-rbhtr\" (UID: \"642b4433-a01d-40ce-a14c-7dd42ab4e837\") " pod="openshift-dns/dns-default-rbhtr" Jul 24 03:19:12.586646 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.586269 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-mnmfg\"" Jul 24 03:19:12.586778 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.586759 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 24 03:19:12.587201 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.586903 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 24 03:19:12.587201 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.587055 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 24 03:19:12.587332 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.587277 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-ts9wj"] Jul 24 03:19:12.603732 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.603714 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5zwrl"] Jul 24 03:19:12.630552 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.630523 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rbhtr"] Jul 24 03:19:12.687681 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.687644 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/642b4433-a01d-40ce-a14c-7dd42ab4e837-tmp-dir\") pod \"dns-default-rbhtr\" (UID: \"642b4433-a01d-40ce-a14c-7dd42ab4e837\") " pod="openshift-dns/dns-default-rbhtr" Jul 24 03:19:12.687771 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.687694 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzwz5\" (UniqueName: \"kubernetes.io/projected/2e237dac-e8fc-4d7c-a87c-5c73b5bc1091-kube-api-access-tzwz5\") pod \"ingress-canary-5zwrl\" (UID: \"2e237dac-e8fc-4d7c-a87c-5c73b5bc1091\") " pod="openshift-ingress-canary/ingress-canary-5zwrl" Jul 24 03:19:12.687821 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.687799 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/642b4433-a01d-40ce-a14c-7dd42ab4e837-config-volume\") pod \"dns-default-rbhtr\" (UID: \"642b4433-a01d-40ce-a14c-7dd42ab4e837\") " pod="openshift-dns/dns-default-rbhtr" Jul 24 03:19:12.687855 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.687838 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l59q\" (UniqueName: \"kubernetes.io/projected/642b4433-a01d-40ce-a14c-7dd42ab4e837-kube-api-access-8l59q\") pod \"dns-default-rbhtr\" (UID: \"642b4433-a01d-40ce-a14c-7dd42ab4e837\") " pod="openshift-dns/dns-default-rbhtr" Jul 24 03:19:12.687904 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.687863 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/642b4433-a01d-40ce-a14c-7dd42ab4e837-metrics-tls\") pod \"dns-default-rbhtr\" (UID: \"642b4433-a01d-40ce-a14c-7dd42ab4e837\") " pod="openshift-dns/dns-default-rbhtr" Jul 24 03:19:12.687957 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.687907 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b4a79ed4-fe4a-407b-b0de-88c3206cd524-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-ts9wj\" (UID: \"b4a79ed4-fe4a-407b-b0de-88c3206cd524\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-ts9wj" Jul 24 03:19:12.687957 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.687936 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/642b4433-a01d-40ce-a14c-7dd42ab4e837-tmp-dir\") pod \"dns-default-rbhtr\" (UID: \"642b4433-a01d-40ce-a14c-7dd42ab4e837\") " pod="openshift-dns/dns-default-rbhtr" Jul 24 03:19:12.688055 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.687959 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2e237dac-e8fc-4d7c-a87c-5c73b5bc1091-cert\") pod \"ingress-canary-5zwrl\" (UID: \"2e237dac-e8fc-4d7c-a87c-5c73b5bc1091\") " pod="openshift-ingress-canary/ingress-canary-5zwrl" Jul 24 03:19:12.688393 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.688373 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/642b4433-a01d-40ce-a14c-7dd42ab4e837-config-volume\") pod \"dns-default-rbhtr\" (UID: \"642b4433-a01d-40ce-a14c-7dd42ab4e837\") " pod="openshift-dns/dns-default-rbhtr" Jul 24 03:19:12.691554 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.691534 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/642b4433-a01d-40ce-a14c-7dd42ab4e837-metrics-tls\") pod \"dns-default-rbhtr\" (UID: \"642b4433-a01d-40ce-a14c-7dd42ab4e837\") " pod="openshift-dns/dns-default-rbhtr" Jul 24 03:19:12.691758 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.691738 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b4a79ed4-fe4a-407b-b0de-88c3206cd524-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-ts9wj\" (UID: \"b4a79ed4-fe4a-407b-b0de-88c3206cd524\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-ts9wj" Jul 24 03:19:12.701728 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.701700 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8l59q\" (UniqueName: \"kubernetes.io/projected/642b4433-a01d-40ce-a14c-7dd42ab4e837-kube-api-access-8l59q\") pod \"dns-default-rbhtr\" (UID: \"642b4433-a01d-40ce-a14c-7dd42ab4e837\") " pod="openshift-dns/dns-default-rbhtr" Jul 24 03:19:12.705225 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.705194 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5t6sr"] Jul 24 03:19:12.707545 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.707529 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5t6sr" Jul 24 03:19:12.710461 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.710443 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tmhzr\"" Jul 24 03:19:12.710548 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.710473 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 24 03:19:12.710851 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.710838 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 24 03:19:12.711035 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.711018 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 24 03:19:12.711485 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.711470 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 24 03:19:12.768145 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.768121 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5t6sr"] Jul 24 03:19:12.788221 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.788171 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cf923b24-c681-4393-b0a9-7db5c0183552-data-volume\") pod \"insights-runtime-extractor-5t6sr\" (UID: \"cf923b24-c681-4393-b0a9-7db5c0183552\") " pod="openshift-insights/insights-runtime-extractor-5t6sr" Jul 24 03:19:12.788221 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.788211 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2e237dac-e8fc-4d7c-a87c-5c73b5bc1091-cert\") pod \"ingress-canary-5zwrl\" (UID: \"2e237dac-e8fc-4d7c-a87c-5c73b5bc1091\") " pod="openshift-ingress-canary/ingress-canary-5zwrl" Jul 24 03:19:12.788346 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.788232 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cf923b24-c681-4393-b0a9-7db5c0183552-crio-socket\") pod \"insights-runtime-extractor-5t6sr\" (UID: \"cf923b24-c681-4393-b0a9-7db5c0183552\") " pod="openshift-insights/insights-runtime-extractor-5t6sr" Jul 24 03:19:12.788346 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.788256 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cf923b24-c681-4393-b0a9-7db5c0183552-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5t6sr\" (UID: \"cf923b24-c681-4393-b0a9-7db5c0183552\") " pod="openshift-insights/insights-runtime-extractor-5t6sr" Jul 24 03:19:12.788346 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.788276 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzwz5\" (UniqueName: \"kubernetes.io/projected/2e237dac-e8fc-4d7c-a87c-5c73b5bc1091-kube-api-access-tzwz5\") pod \"ingress-canary-5zwrl\" (UID: \"2e237dac-e8fc-4d7c-a87c-5c73b5bc1091\") " pod="openshift-ingress-canary/ingress-canary-5zwrl" Jul 24 03:19:12.788493 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.788386 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cf923b24-c681-4393-b0a9-7db5c0183552-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5t6sr\" (UID: \"cf923b24-c681-4393-b0a9-7db5c0183552\") " pod="openshift-insights/insights-runtime-extractor-5t6sr" Jul 24 03:19:12.788493 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.788463 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ms68j\" (UniqueName: \"kubernetes.io/projected/cf923b24-c681-4393-b0a9-7db5c0183552-kube-api-access-ms68j\") pod \"insights-runtime-extractor-5t6sr\" (UID: \"cf923b24-c681-4393-b0a9-7db5c0183552\") " pod="openshift-insights/insights-runtime-extractor-5t6sr" Jul 24 03:19:12.790339 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.790322 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2e237dac-e8fc-4d7c-a87c-5c73b5bc1091-cert\") pod \"ingress-canary-5zwrl\" (UID: \"2e237dac-e8fc-4d7c-a87c-5c73b5bc1091\") " pod="openshift-ingress-canary/ingress-canary-5zwrl" Jul 24 03:19:12.803161 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.803140 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzwz5\" (UniqueName: \"kubernetes.io/projected/2e237dac-e8fc-4d7c-a87c-5c73b5bc1091-kube-api-access-tzwz5\") pod \"ingress-canary-5zwrl\" (UID: \"2e237dac-e8fc-4d7c-a87c-5c73b5bc1091\") " pod="openshift-ingress-canary/ingress-canary-5zwrl" Jul 24 03:19:12.886882 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.886859 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-ts9wj" Jul 24 03:19:12.889612 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.889596 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cf923b24-c681-4393-b0a9-7db5c0183552-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5t6sr\" (UID: \"cf923b24-c681-4393-b0a9-7db5c0183552\") " pod="openshift-insights/insights-runtime-extractor-5t6sr" Jul 24 03:19:12.889689 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.889638 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms68j\" (UniqueName: \"kubernetes.io/projected/cf923b24-c681-4393-b0a9-7db5c0183552-kube-api-access-ms68j\") pod \"insights-runtime-extractor-5t6sr\" (UID: \"cf923b24-c681-4393-b0a9-7db5c0183552\") " pod="openshift-insights/insights-runtime-extractor-5t6sr" Jul 24 03:19:12.889742 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.889693 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cf923b24-c681-4393-b0a9-7db5c0183552-data-volume\") pod \"insights-runtime-extractor-5t6sr\" (UID: \"cf923b24-c681-4393-b0a9-7db5c0183552\") " pod="openshift-insights/insights-runtime-extractor-5t6sr" Jul 24 03:19:12.889742 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.889719 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cf923b24-c681-4393-b0a9-7db5c0183552-crio-socket\") pod \"insights-runtime-extractor-5t6sr\" (UID: \"cf923b24-c681-4393-b0a9-7db5c0183552\") " pod="openshift-insights/insights-runtime-extractor-5t6sr" Jul 24 03:19:12.889742 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.889735 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cf923b24-c681-4393-b0a9-7db5c0183552-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5t6sr\" (UID: \"cf923b24-c681-4393-b0a9-7db5c0183552\") " pod="openshift-insights/insights-runtime-extractor-5t6sr" Jul 24 03:19:12.889928 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.889904 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cf923b24-c681-4393-b0a9-7db5c0183552-crio-socket\") pod \"insights-runtime-extractor-5t6sr\" (UID: \"cf923b24-c681-4393-b0a9-7db5c0183552\") " pod="openshift-insights/insights-runtime-extractor-5t6sr" Jul 24 03:19:12.890035 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.890020 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cf923b24-c681-4393-b0a9-7db5c0183552-data-volume\") pod \"insights-runtime-extractor-5t6sr\" (UID: \"cf923b24-c681-4393-b0a9-7db5c0183552\") " pod="openshift-insights/insights-runtime-extractor-5t6sr" Jul 24 03:19:12.890214 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.890199 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cf923b24-c681-4393-b0a9-7db5c0183552-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5t6sr\" (UID: \"cf923b24-c681-4393-b0a9-7db5c0183552\") " pod="openshift-insights/insights-runtime-extractor-5t6sr" Jul 24 03:19:12.891858 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.891844 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cf923b24-c681-4393-b0a9-7db5c0183552-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5t6sr\" (UID: \"cf923b24-c681-4393-b0a9-7db5c0183552\") " pod="openshift-insights/insights-runtime-extractor-5t6sr" Jul 24 03:19:12.894822 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.894804 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rbhtr" Jul 24 03:19:12.899126 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.899108 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5zwrl" Jul 24 03:19:12.923032 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.922849 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-9h4l2"] Jul 24 03:19:12.926470 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.926433 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-9h4l2" Jul 24 03:19:12.926470 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.926460 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms68j\" (UniqueName: \"kubernetes.io/projected/cf923b24-c681-4393-b0a9-7db5c0183552-kube-api-access-ms68j\") pod \"insights-runtime-extractor-5t6sr\" (UID: \"cf923b24-c681-4393-b0a9-7db5c0183552\") " pod="openshift-insights/insights-runtime-extractor-5t6sr" Jul 24 03:19:12.930995 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.930977 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-gvbt8\"" Jul 24 03:19:12.932880 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.931905 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 24 03:19:12.932880 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.932172 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 24 03:19:12.947992 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.947966 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-9h4l2"] Jul 24 03:19:12.990279 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:12.990103 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pq7c\" (UniqueName: \"kubernetes.io/projected/18fbf1b4-55a4-4e8c-806e-408fb712a12a-kube-api-access-7pq7c\") pod \"downloads-7c97dc4555-9h4l2\" (UID: \"18fbf1b4-55a4-4e8c-806e-408fb712a12a\") " pod="openshift-console/downloads-7c97dc4555-9h4l2" Jul 24 03:19:13.016042 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:13.015995 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5t6sr" Jul 24 03:19:13.040525 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:13.040463 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-ts9wj"] Jul 24 03:19:13.043728 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:19:13.043693 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4a79ed4_fe4a_407b_b0de_88c3206cd524.slice/crio-e7fbb714d185dfa54fac563e65a5117b0fe24db303af6a34946bc4fbbcad6930 WatchSource:0}: Error finding container e7fbb714d185dfa54fac563e65a5117b0fe24db303af6a34946bc4fbbcad6930: Status 404 returned error can't find the container with id e7fbb714d185dfa54fac563e65a5117b0fe24db303af6a34946bc4fbbcad6930 Jul 24 03:19:13.055198 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:13.055173 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rbhtr"] Jul 24 03:19:13.058354 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:19:13.058334 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod642b4433_a01d_40ce_a14c_7dd42ab4e837.slice/crio-f411155c14ef45b2b4dc02ecdb1ef958d95cbad5f83e1cf05bd31888b186ed13 WatchSource:0}: Error finding container f411155c14ef45b2b4dc02ecdb1ef958d95cbad5f83e1cf05bd31888b186ed13: Status 404 returned error can't find the container with id f411155c14ef45b2b4dc02ecdb1ef958d95cbad5f83e1cf05bd31888b186ed13 Jul 24 03:19:13.073240 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:13.073176 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5zwrl"] Jul 24 03:19:13.101927 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:13.101541 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pq7c\" (UniqueName: \"kubernetes.io/projected/18fbf1b4-55a4-4e8c-806e-408fb712a12a-kube-api-access-7pq7c\") pod \"downloads-7c97dc4555-9h4l2\" (UID: \"18fbf1b4-55a4-4e8c-806e-408fb712a12a\") " pod="openshift-console/downloads-7c97dc4555-9h4l2" Jul 24 03:19:13.110190 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:13.110171 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pq7c\" (UniqueName: \"kubernetes.io/projected/18fbf1b4-55a4-4e8c-806e-408fb712a12a-kube-api-access-7pq7c\") pod \"downloads-7c97dc4555-9h4l2\" (UID: \"18fbf1b4-55a4-4e8c-806e-408fb712a12a\") " pod="openshift-console/downloads-7c97dc4555-9h4l2" Jul 24 03:19:13.166335 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:13.166303 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5t6sr"] Jul 24 03:19:13.168817 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:19:13.168788 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf923b24_c681_4393_b0a9_7db5c0183552.slice/crio-68158b76798bf383c32fd94ab12d2c37a28e92666ae5f833600bc8604a7a37cb WatchSource:0}: Error finding container 68158b76798bf383c32fd94ab12d2c37a28e92666ae5f833600bc8604a7a37cb: Status 404 returned error can't find the container with id 68158b76798bf383c32fd94ab12d2c37a28e92666ae5f833600bc8604a7a37cb Jul 24 03:19:13.238360 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:13.238335 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-9h4l2" Jul 24 03:19:13.355897 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:13.355867 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-9h4l2"] Jul 24 03:19:13.360262 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:19:13.360238 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18fbf1b4_55a4_4e8c_806e_408fb712a12a.slice/crio-469307065f8c083672d0d4350d03843421b5cf44f8e053c90a4f297269c3576b WatchSource:0}: Error finding container 469307065f8c083672d0d4350d03843421b5cf44f8e053c90a4f297269c3576b: Status 404 returned error can't find the container with id 469307065f8c083672d0d4350d03843421b5cf44f8e053c90a4f297269c3576b Jul 24 03:19:13.516938 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:13.516902 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:19:13.518538 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:13.518511 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:19:13.518683 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:13.518590 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:19:13.521178 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:13.521159 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 24 03:19:13.521518 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:13.521501 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 24 03:19:13.521699 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:13.521686 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-w5g4d\"" Jul 24 03:19:13.521864 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:13.521851 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 24 03:19:13.522109 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:13.522096 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 24 03:19:13.522262 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:13.522248 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ph9hg\"" Jul 24 03:19:13.857595 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:13.857510 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-9h4l2" event={"ID":"18fbf1b4-55a4-4e8c-806e-408fb712a12a","Type":"ContainerStarted","Data":"469307065f8c083672d0d4350d03843421b5cf44f8e053c90a4f297269c3576b"} Jul 24 03:19:13.859812 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:13.859781 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-ts9wj" event={"ID":"b4a79ed4-fe4a-407b-b0de-88c3206cd524","Type":"ContainerStarted","Data":"e7fbb714d185dfa54fac563e65a5117b0fe24db303af6a34946bc4fbbcad6930"} Jul 24 03:19:13.866705 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:13.866435 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rbhtr" event={"ID":"642b4433-a01d-40ce-a14c-7dd42ab4e837","Type":"ContainerStarted","Data":"f411155c14ef45b2b4dc02ecdb1ef958d95cbad5f83e1cf05bd31888b186ed13"} Jul 24 03:19:13.879579 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:13.879553 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5t6sr" event={"ID":"cf923b24-c681-4393-b0a9-7db5c0183552","Type":"ContainerStarted","Data":"68158b76798bf383c32fd94ab12d2c37a28e92666ae5f833600bc8604a7a37cb"} Jul 24 03:19:13.884156 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:13.884099 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5zwrl" event={"ID":"2e237dac-e8fc-4d7c-a87c-5c73b5bc1091","Type":"ContainerStarted","Data":"72a58692664a350fd8e4652c760c7430885b29032338d48dbe07a6486c8733c6"} Jul 24 03:19:14.892250 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:14.889816 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-ts9wj" event={"ID":"b4a79ed4-fe4a-407b-b0de-88c3206cd524","Type":"ContainerStarted","Data":"52eda5d635acfe27f27273f46c6243d7e7bb22e5c560dd4c9f25636192bd86b1"} Jul 24 03:19:14.892250 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:14.892185 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-ts9wj" Jul 24 03:19:14.897722 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:14.897699 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-ts9wj" Jul 24 03:19:14.926628 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:14.926407 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-ts9wj" podStartSLOduration=1.317971294 podStartE2EDuration="2.926388749s" totalImagesPullingTime="1.608417455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:19:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:19:14.90717319 +0000 UTC m=+52.182744541" watchObservedRunningTime="2026-07-24 03:19:14.926388749 +0000 UTC m=+52.201960093" Jul 24 03:19:15.632677 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:15.631913 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-qsrcp"] Jul 24 03:19:15.634179 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:15.634153 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-qsrcp" Jul 24 03:19:15.636983 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:15.636959 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 24 03:19:15.637249 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:15.637232 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-hrzms\"" Jul 24 03:19:15.637919 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:15.637525 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 24 03:19:15.637919 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:15.637555 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 24 03:19:15.637919 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:15.637771 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 24 03:19:15.638407 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:15.638247 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 24 03:19:15.646437 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:15.646020 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-qsrcp"] Jul 24 03:19:15.722677 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:15.722625 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/27c2b435-24ce-4119-a81f-3aae0aa7d911-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-qsrcp\" (UID: \"27c2b435-24ce-4119-a81f-3aae0aa7d911\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-qsrcp" Jul 24 03:19:15.722823 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:15.722688 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/27c2b435-24ce-4119-a81f-3aae0aa7d911-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-qsrcp\" (UID: \"27c2b435-24ce-4119-a81f-3aae0aa7d911\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-qsrcp" Jul 24 03:19:15.722823 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:15.722722 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/27c2b435-24ce-4119-a81f-3aae0aa7d911-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-qsrcp\" (UID: \"27c2b435-24ce-4119-a81f-3aae0aa7d911\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-qsrcp" Jul 24 03:19:15.722823 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:15.722750 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wx9sw\" (UniqueName: \"kubernetes.io/projected/27c2b435-24ce-4119-a81f-3aae0aa7d911-kube-api-access-wx9sw\") pod \"prometheus-operator-84d69f794d-qsrcp\" (UID: \"27c2b435-24ce-4119-a81f-3aae0aa7d911\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-qsrcp" Jul 24 03:19:15.823828 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:15.823196 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/27c2b435-24ce-4119-a81f-3aae0aa7d911-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-qsrcp\" (UID: \"27c2b435-24ce-4119-a81f-3aae0aa7d911\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-qsrcp" Jul 24 03:19:15.823828 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:15.823242 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/27c2b435-24ce-4119-a81f-3aae0aa7d911-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-qsrcp\" (UID: \"27c2b435-24ce-4119-a81f-3aae0aa7d911\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-qsrcp" Jul 24 03:19:15.823828 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:15.823308 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/27c2b435-24ce-4119-a81f-3aae0aa7d911-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-qsrcp\" (UID: \"27c2b435-24ce-4119-a81f-3aae0aa7d911\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-qsrcp" Jul 24 03:19:15.823828 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:15.823338 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wx9sw\" (UniqueName: \"kubernetes.io/projected/27c2b435-24ce-4119-a81f-3aae0aa7d911-kube-api-access-wx9sw\") pod \"prometheus-operator-84d69f794d-qsrcp\" (UID: \"27c2b435-24ce-4119-a81f-3aae0aa7d911\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-qsrcp" Jul 24 03:19:15.824164 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:15.824035 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/27c2b435-24ce-4119-a81f-3aae0aa7d911-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-qsrcp\" (UID: \"27c2b435-24ce-4119-a81f-3aae0aa7d911\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-qsrcp" Jul 24 03:19:15.827101 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:15.827055 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/27c2b435-24ce-4119-a81f-3aae0aa7d911-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-qsrcp\" (UID: \"27c2b435-24ce-4119-a81f-3aae0aa7d911\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-qsrcp" Jul 24 03:19:15.828223 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:15.828196 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/27c2b435-24ce-4119-a81f-3aae0aa7d911-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-qsrcp\" (UID: \"27c2b435-24ce-4119-a81f-3aae0aa7d911\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-qsrcp" Jul 24 03:19:15.831495 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:15.831473 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wx9sw\" (UniqueName: \"kubernetes.io/projected/27c2b435-24ce-4119-a81f-3aae0aa7d911-kube-api-access-wx9sw\") pod \"prometheus-operator-84d69f794d-qsrcp\" (UID: \"27c2b435-24ce-4119-a81f-3aae0aa7d911\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-qsrcp" Jul 24 03:19:15.946980 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:15.946903 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-qsrcp" Jul 24 03:19:19.983788 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:19.983734 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-qsrcp"] Jul 24 03:19:19.988515 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:19:19.988476 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27c2b435_24ce_4119_a81f_3aae0aa7d911.slice/crio-47d3c0006bcdb764e7c49173d414326f848366a2d4971611903840a79106d0ef WatchSource:0}: Error finding container 47d3c0006bcdb764e7c49173d414326f848366a2d4971611903840a79106d0ef: Status 404 returned error can't find the container with id 47d3c0006bcdb764e7c49173d414326f848366a2d4971611903840a79106d0ef Jul 24 03:19:20.914435 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:20.914344 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-qsrcp" event={"ID":"27c2b435-24ce-4119-a81f-3aae0aa7d911","Type":"ContainerStarted","Data":"47d3c0006bcdb764e7c49173d414326f848366a2d4971611903840a79106d0ef"} Jul 24 03:19:20.918030 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:20.917245 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rbhtr" event={"ID":"642b4433-a01d-40ce-a14c-7dd42ab4e837","Type":"ContainerStarted","Data":"d1dcf0fe51755739c6f8a0d814184ece177ad716c5fca9724e5a54714dfdb4e4"} Jul 24 03:19:20.918030 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:20.917296 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rbhtr" event={"ID":"642b4433-a01d-40ce-a14c-7dd42ab4e837","Type":"ContainerStarted","Data":"b4e81218b7cb83c9f834cfcbbeaf4783106ec2a421290d0e81ac296b85f9a708"} Jul 24 03:19:20.918237 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:20.918070 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-rbhtr" Jul 24 03:19:20.920188 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:20.920124 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5t6sr" event={"ID":"cf923b24-c681-4393-b0a9-7db5c0183552","Type":"ContainerStarted","Data":"ad1cd8aba3bf2e538f1084480dd3c13e8753ed831da0f0fa0db06d45479eb09d"} Jul 24 03:19:20.922596 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:20.922571 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5zwrl" event={"ID":"2e237dac-e8fc-4d7c-a87c-5c73b5bc1091","Type":"ContainerStarted","Data":"b40dc2807640a895f900972b6ecf4201a2fd23ecf1b9c10f9c3b42bdef1bad40"} Jul 24 03:19:20.938597 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:20.936231 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-rbhtr" podStartSLOduration=2.154336818 podStartE2EDuration="8.936065825s" totalImagesPullingTime="6.781729007s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:19:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:19:20.934189994 +0000 UTC m=+58.209761389" watchObservedRunningTime="2026-07-24 03:19:20.936065825 +0000 UTC m=+58.211637179" Jul 24 03:19:20.954565 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:20.954395 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5zwrl" podStartSLOduration=2.197224134 podStartE2EDuration="8.954380171s" totalImagesPullingTime="6.757156037s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:19:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:19:20.953775449 +0000 UTC m=+58.229346800" watchObservedRunningTime="2026-07-24 03:19:20.954380171 +0000 UTC m=+58.229951524" Jul 24 03:19:21.929394 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:21.929355 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-qsrcp" event={"ID":"27c2b435-24ce-4119-a81f-3aae0aa7d911","Type":"ContainerStarted","Data":"8bc794f4c9c5dea0635957b6316497d67361be721d3192709b073ca52205b298"} Jul 24 03:19:21.929871 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:21.929424 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-qsrcp" event={"ID":"27c2b435-24ce-4119-a81f-3aae0aa7d911","Type":"ContainerStarted","Data":"6cecd54ec811474b178c4aaa48fd1df4910f48eab1bf409166bb26996b059bc5"} Jul 24 03:19:21.950493 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:21.950295 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-qsrcp" podStartSLOduration=5.488003863 podStartE2EDuration="6.950273071s" totalImagesPullingTime="1.462269208s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:19:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:19:21.948274668 +0000 UTC m=+59.223846018" watchObservedRunningTime="2026-07-24 03:19:21.950273071 +0000 UTC m=+59.225844424" Jul 24 03:19:21.994413 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:21.994370 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-797fc79d89-jj4f4"] Jul 24 03:19:22.000909 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:22.000878 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-797fc79d89-jj4f4" Jul 24 03:19:22.003338 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:22.003309 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 24 03:19:22.003482 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:22.003457 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 24 03:19:22.003769 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:22.003726 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 24 03:19:22.003769 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:22.003742 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 24 03:19:22.004174 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:22.003991 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-mvmn5\"" Jul 24 03:19:22.004174 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:22.003993 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 24 03:19:22.010090 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:22.010068 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-797fc79d89-jj4f4"] Jul 24 03:19:22.072582 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:22.072544 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a32fb5b0-3f50-486a-9f5b-f989b67cd313-console-serving-cert\") pod \"console-797fc79d89-jj4f4\" (UID: \"a32fb5b0-3f50-486a-9f5b-f989b67cd313\") " pod="openshift-console/console-797fc79d89-jj4f4" Jul 24 03:19:22.072761 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:22.072677 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a32fb5b0-3f50-486a-9f5b-f989b67cd313-console-config\") pod \"console-797fc79d89-jj4f4\" (UID: \"a32fb5b0-3f50-486a-9f5b-f989b67cd313\") " pod="openshift-console/console-797fc79d89-jj4f4" Jul 24 03:19:22.072761 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:22.072723 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a32fb5b0-3f50-486a-9f5b-f989b67cd313-oauth-serving-cert\") pod \"console-797fc79d89-jj4f4\" (UID: \"a32fb5b0-3f50-486a-9f5b-f989b67cd313\") " pod="openshift-console/console-797fc79d89-jj4f4" Jul 24 03:19:22.072876 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:22.072783 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a32fb5b0-3f50-486a-9f5b-f989b67cd313-service-ca\") pod \"console-797fc79d89-jj4f4\" (UID: \"a32fb5b0-3f50-486a-9f5b-f989b67cd313\") " pod="openshift-console/console-797fc79d89-jj4f4" Jul 24 03:19:22.072876 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:22.072854 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4dh9\" (UniqueName: \"kubernetes.io/projected/a32fb5b0-3f50-486a-9f5b-f989b67cd313-kube-api-access-p4dh9\") pod \"console-797fc79d89-jj4f4\" (UID: \"a32fb5b0-3f50-486a-9f5b-f989b67cd313\") " pod="openshift-console/console-797fc79d89-jj4f4" Jul 24 03:19:22.072978 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:22.072884 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a32fb5b0-3f50-486a-9f5b-f989b67cd313-console-oauth-config\") pod \"console-797fc79d89-jj4f4\" (UID: \"a32fb5b0-3f50-486a-9f5b-f989b67cd313\") " pod="openshift-console/console-797fc79d89-jj4f4" Jul 24 03:19:22.173813 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:22.173775 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a32fb5b0-3f50-486a-9f5b-f989b67cd313-console-serving-cert\") pod \"console-797fc79d89-jj4f4\" (UID: \"a32fb5b0-3f50-486a-9f5b-f989b67cd313\") " pod="openshift-console/console-797fc79d89-jj4f4" Jul 24 03:19:22.173992 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:22.173937 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a32fb5b0-3f50-486a-9f5b-f989b67cd313-console-config\") pod \"console-797fc79d89-jj4f4\" (UID: \"a32fb5b0-3f50-486a-9f5b-f989b67cd313\") " pod="openshift-console/console-797fc79d89-jj4f4" Jul 24 03:19:22.173992 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:22.173973 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a32fb5b0-3f50-486a-9f5b-f989b67cd313-oauth-serving-cert\") pod \"console-797fc79d89-jj4f4\" (UID: \"a32fb5b0-3f50-486a-9f5b-f989b67cd313\") " pod="openshift-console/console-797fc79d89-jj4f4" Jul 24 03:19:22.174104 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:22.174039 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a32fb5b0-3f50-486a-9f5b-f989b67cd313-service-ca\") pod \"console-797fc79d89-jj4f4\" (UID: \"a32fb5b0-3f50-486a-9f5b-f989b67cd313\") " pod="openshift-console/console-797fc79d89-jj4f4" Jul 24 03:19:22.174104 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:22.174085 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4dh9\" (UniqueName: \"kubernetes.io/projected/a32fb5b0-3f50-486a-9f5b-f989b67cd313-kube-api-access-p4dh9\") pod \"console-797fc79d89-jj4f4\" (UID: \"a32fb5b0-3f50-486a-9f5b-f989b67cd313\") " pod="openshift-console/console-797fc79d89-jj4f4" Jul 24 03:19:22.174194 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:22.174113 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a32fb5b0-3f50-486a-9f5b-f989b67cd313-console-oauth-config\") pod \"console-797fc79d89-jj4f4\" (UID: \"a32fb5b0-3f50-486a-9f5b-f989b67cd313\") " pod="openshift-console/console-797fc79d89-jj4f4" Jul 24 03:19:22.174739 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:22.174654 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a32fb5b0-3f50-486a-9f5b-f989b67cd313-console-config\") pod \"console-797fc79d89-jj4f4\" (UID: \"a32fb5b0-3f50-486a-9f5b-f989b67cd313\") " pod="openshift-console/console-797fc79d89-jj4f4" Jul 24 03:19:22.174882 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:22.174861 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a32fb5b0-3f50-486a-9f5b-f989b67cd313-service-ca\") pod \"console-797fc79d89-jj4f4\" (UID: \"a32fb5b0-3f50-486a-9f5b-f989b67cd313\") " pod="openshift-console/console-797fc79d89-jj4f4" Jul 24 03:19:22.175173 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:22.175129 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a32fb5b0-3f50-486a-9f5b-f989b67cd313-oauth-serving-cert\") pod \"console-797fc79d89-jj4f4\" (UID: \"a32fb5b0-3f50-486a-9f5b-f989b67cd313\") " pod="openshift-console/console-797fc79d89-jj4f4" Jul 24 03:19:22.176734 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:22.176715 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a32fb5b0-3f50-486a-9f5b-f989b67cd313-console-oauth-config\") pod \"console-797fc79d89-jj4f4\" (UID: \"a32fb5b0-3f50-486a-9f5b-f989b67cd313\") " pod="openshift-console/console-797fc79d89-jj4f4" Jul 24 03:19:22.176821 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:22.176762 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a32fb5b0-3f50-486a-9f5b-f989b67cd313-console-serving-cert\") pod \"console-797fc79d89-jj4f4\" (UID: \"a32fb5b0-3f50-486a-9f5b-f989b67cd313\") " pod="openshift-console/console-797fc79d89-jj4f4" Jul 24 03:19:22.184134 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:22.184060 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4dh9\" (UniqueName: \"kubernetes.io/projected/a32fb5b0-3f50-486a-9f5b-f989b67cd313-kube-api-access-p4dh9\") pod \"console-797fc79d89-jj4f4\" (UID: \"a32fb5b0-3f50-486a-9f5b-f989b67cd313\") " pod="openshift-console/console-797fc79d89-jj4f4" Jul 24 03:19:22.312773 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:22.312738 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-797fc79d89-jj4f4" Jul 24 03:19:23.992187 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:23.992152 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-sgn88"] Jul 24 03:19:24.099943 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.099905 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-hkbh6"] Jul 24 03:19:24.099943 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.100004 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-sgn88" Jul 24 03:19:24.102554 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.102532 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 24 03:19:24.102812 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.102786 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-c2qsd\"" Jul 24 03:19:24.102907 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.102840 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 24 03:19:24.133827 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.133797 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-sgn88"] Jul 24 03:19:24.133972 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.133947 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hkbh6" Jul 24 03:19:24.136400 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.136379 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 24 03:19:24.136503 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.136421 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 24 03:19:24.136503 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.136472 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 24 03:19:24.136754 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.136733 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-rs2nn\"" Jul 24 03:19:24.194358 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.194325 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lg5s\" (UniqueName: \"kubernetes.io/projected/28810c1c-0d61-437d-b68d-79d6c933120c-kube-api-access-7lg5s\") pod \"node-exporter-hkbh6\" (UID: \"28810c1c-0d61-437d-b68d-79d6c933120c\") " pod="openshift-monitoring/node-exporter-hkbh6" Jul 24 03:19:24.194522 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.194387 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vf7nh\" (UniqueName: \"kubernetes.io/projected/3ae74270-00fd-466b-9359-389661481fff-kube-api-access-vf7nh\") pod \"openshift-state-metrics-b8f49f5c6-sgn88\" (UID: \"3ae74270-00fd-466b-9359-389661481fff\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-sgn88" Jul 24 03:19:24.194522 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.194463 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/28810c1c-0d61-437d-b68d-79d6c933120c-node-exporter-textfile\") pod \"node-exporter-hkbh6\" (UID: \"28810c1c-0d61-437d-b68d-79d6c933120c\") " pod="openshift-monitoring/node-exporter-hkbh6" Jul 24 03:19:24.194619 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.194522 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28810c1c-0d61-437d-b68d-79d6c933120c-metrics-client-ca\") pod \"node-exporter-hkbh6\" (UID: \"28810c1c-0d61-437d-b68d-79d6c933120c\") " pod="openshift-monitoring/node-exporter-hkbh6" Jul 24 03:19:24.194619 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.194549 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/28810c1c-0d61-437d-b68d-79d6c933120c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hkbh6\" (UID: \"28810c1c-0d61-437d-b68d-79d6c933120c\") " pod="openshift-monitoring/node-exporter-hkbh6" Jul 24 03:19:24.194619 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.194601 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ae74270-00fd-466b-9359-389661481fff-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-sgn88\" (UID: \"3ae74270-00fd-466b-9359-389661481fff\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-sgn88" Jul 24 03:19:24.194774 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.194623 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/28810c1c-0d61-437d-b68d-79d6c933120c-sys\") pod \"node-exporter-hkbh6\" (UID: \"28810c1c-0d61-437d-b68d-79d6c933120c\") " pod="openshift-monitoring/node-exporter-hkbh6" Jul 24 03:19:24.194774 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.194652 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/28810c1c-0d61-437d-b68d-79d6c933120c-node-exporter-tls\") pod \"node-exporter-hkbh6\" (UID: \"28810c1c-0d61-437d-b68d-79d6c933120c\") " pod="openshift-monitoring/node-exporter-hkbh6" Jul 24 03:19:24.194774 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.194734 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/28810c1c-0d61-437d-b68d-79d6c933120c-node-exporter-accelerators-collector-config\") pod \"node-exporter-hkbh6\" (UID: \"28810c1c-0d61-437d-b68d-79d6c933120c\") " pod="openshift-monitoring/node-exporter-hkbh6" Jul 24 03:19:24.194888 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.194776 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ae74270-00fd-466b-9359-389661481fff-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-sgn88\" (UID: \"3ae74270-00fd-466b-9359-389661481fff\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-sgn88" Jul 24 03:19:24.194888 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.194796 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/28810c1c-0d61-437d-b68d-79d6c933120c-root\") pod \"node-exporter-hkbh6\" (UID: \"28810c1c-0d61-437d-b68d-79d6c933120c\") " pod="openshift-monitoring/node-exporter-hkbh6" Jul 24 03:19:24.194888 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.194812 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/28810c1c-0d61-437d-b68d-79d6c933120c-node-exporter-wtmp\") pod \"node-exporter-hkbh6\" (UID: \"28810c1c-0d61-437d-b68d-79d6c933120c\") " pod="openshift-monitoring/node-exporter-hkbh6" Jul 24 03:19:24.194981 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.194917 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3ae74270-00fd-466b-9359-389661481fff-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-sgn88\" (UID: \"3ae74270-00fd-466b-9359-389661481fff\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-sgn88" Jul 24 03:19:24.296250 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.296140 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3ae74270-00fd-466b-9359-389661481fff-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-sgn88\" (UID: \"3ae74270-00fd-466b-9359-389661481fff\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-sgn88" Jul 24 03:19:24.296250 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.296223 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lg5s\" (UniqueName: \"kubernetes.io/projected/28810c1c-0d61-437d-b68d-79d6c933120c-kube-api-access-7lg5s\") pod \"node-exporter-hkbh6\" (UID: \"28810c1c-0d61-437d-b68d-79d6c933120c\") " pod="openshift-monitoring/node-exporter-hkbh6" Jul 24 03:19:24.296465 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.296282 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vf7nh\" (UniqueName: \"kubernetes.io/projected/3ae74270-00fd-466b-9359-389661481fff-kube-api-access-vf7nh\") pod \"openshift-state-metrics-b8f49f5c6-sgn88\" (UID: \"3ae74270-00fd-466b-9359-389661481fff\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-sgn88" Jul 24 03:19:24.296465 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.296315 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/28810c1c-0d61-437d-b68d-79d6c933120c-node-exporter-textfile\") pod \"node-exporter-hkbh6\" (UID: \"28810c1c-0d61-437d-b68d-79d6c933120c\") " pod="openshift-monitoring/node-exporter-hkbh6" Jul 24 03:19:24.296465 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:19:24.296348 2579 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 24 03:19:24.296465 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.296369 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28810c1c-0d61-437d-b68d-79d6c933120c-metrics-client-ca\") pod \"node-exporter-hkbh6\" (UID: \"28810c1c-0d61-437d-b68d-79d6c933120c\") " pod="openshift-monitoring/node-exporter-hkbh6" Jul 24 03:19:24.296465 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.296396 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/28810c1c-0d61-437d-b68d-79d6c933120c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hkbh6\" (UID: \"28810c1c-0d61-437d-b68d-79d6c933120c\") " pod="openshift-monitoring/node-exporter-hkbh6" Jul 24 03:19:24.296465 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:19:24.296438 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ae74270-00fd-466b-9359-389661481fff-openshift-state-metrics-tls podName:3ae74270-00fd-466b-9359-389661481fff nodeName:}" failed. No retries permitted until 2026-07-24 03:19:24.7964164 +0000 UTC m=+62.071987731 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/3ae74270-00fd-466b-9359-389661481fff-openshift-state-metrics-tls") pod "openshift-state-metrics-b8f49f5c6-sgn88" (UID: "3ae74270-00fd-466b-9359-389661481fff") : secret "openshift-state-metrics-tls" not found Jul 24 03:19:24.296796 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.296467 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ae74270-00fd-466b-9359-389661481fff-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-sgn88\" (UID: \"3ae74270-00fd-466b-9359-389661481fff\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-sgn88" Jul 24 03:19:24.296796 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.296510 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/28810c1c-0d61-437d-b68d-79d6c933120c-sys\") pod \"node-exporter-hkbh6\" (UID: \"28810c1c-0d61-437d-b68d-79d6c933120c\") " pod="openshift-monitoring/node-exporter-hkbh6" Jul 24 03:19:24.296796 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.296550 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/28810c1c-0d61-437d-b68d-79d6c933120c-node-exporter-tls\") pod \"node-exporter-hkbh6\" (UID: \"28810c1c-0d61-437d-b68d-79d6c933120c\") " pod="openshift-monitoring/node-exporter-hkbh6" Jul 24 03:19:24.296796 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.296607 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/28810c1c-0d61-437d-b68d-79d6c933120c-node-exporter-accelerators-collector-config\") pod \"node-exporter-hkbh6\" (UID: \"28810c1c-0d61-437d-b68d-79d6c933120c\") " pod="openshift-monitoring/node-exporter-hkbh6" Jul 24 03:19:24.296796 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.296647 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ae74270-00fd-466b-9359-389661481fff-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-sgn88\" (UID: \"3ae74270-00fd-466b-9359-389661481fff\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-sgn88" Jul 24 03:19:24.296796 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.296701 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/28810c1c-0d61-437d-b68d-79d6c933120c-root\") pod \"node-exporter-hkbh6\" (UID: \"28810c1c-0d61-437d-b68d-79d6c933120c\") " pod="openshift-monitoring/node-exporter-hkbh6" Jul 24 03:19:24.296796 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.296731 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/28810c1c-0d61-437d-b68d-79d6c933120c-node-exporter-wtmp\") pod \"node-exporter-hkbh6\" (UID: \"28810c1c-0d61-437d-b68d-79d6c933120c\") " pod="openshift-monitoring/node-exporter-hkbh6" Jul 24 03:19:24.296796 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.296761 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/28810c1c-0d61-437d-b68d-79d6c933120c-node-exporter-textfile\") pod \"node-exporter-hkbh6\" (UID: \"28810c1c-0d61-437d-b68d-79d6c933120c\") " pod="openshift-monitoring/node-exporter-hkbh6" Jul 24 03:19:24.297158 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.296927 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/28810c1c-0d61-437d-b68d-79d6c933120c-node-exporter-wtmp\") pod \"node-exporter-hkbh6\" (UID: \"28810c1c-0d61-437d-b68d-79d6c933120c\") " pod="openshift-monitoring/node-exporter-hkbh6" Jul 24 03:19:24.297158 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.296977 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/28810c1c-0d61-437d-b68d-79d6c933120c-root\") pod \"node-exporter-hkbh6\" (UID: \"28810c1c-0d61-437d-b68d-79d6c933120c\") " pod="openshift-monitoring/node-exporter-hkbh6" Jul 24 03:19:24.297158 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.297019 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/28810c1c-0d61-437d-b68d-79d6c933120c-sys\") pod \"node-exporter-hkbh6\" (UID: \"28810c1c-0d61-437d-b68d-79d6c933120c\") " pod="openshift-monitoring/node-exporter-hkbh6" Jul 24 03:19:24.297158 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.297129 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28810c1c-0d61-437d-b68d-79d6c933120c-metrics-client-ca\") pod \"node-exporter-hkbh6\" (UID: \"28810c1c-0d61-437d-b68d-79d6c933120c\") " pod="openshift-monitoring/node-exporter-hkbh6" Jul 24 03:19:24.297353 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.297245 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ae74270-00fd-466b-9359-389661481fff-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-sgn88\" (UID: \"3ae74270-00fd-466b-9359-389661481fff\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-sgn88" Jul 24 03:19:24.297601 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.297580 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/28810c1c-0d61-437d-b68d-79d6c933120c-node-exporter-accelerators-collector-config\") pod \"node-exporter-hkbh6\" (UID: \"28810c1c-0d61-437d-b68d-79d6c933120c\") " pod="openshift-monitoring/node-exporter-hkbh6" Jul 24 03:19:24.299742 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.299714 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ae74270-00fd-466b-9359-389661481fff-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-sgn88\" (UID: \"3ae74270-00fd-466b-9359-389661481fff\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-sgn88" Jul 24 03:19:24.299845 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.299799 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/28810c1c-0d61-437d-b68d-79d6c933120c-node-exporter-tls\") pod \"node-exporter-hkbh6\" (UID: \"28810c1c-0d61-437d-b68d-79d6c933120c\") " pod="openshift-monitoring/node-exporter-hkbh6" Jul 24 03:19:24.300139 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.300101 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/28810c1c-0d61-437d-b68d-79d6c933120c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hkbh6\" (UID: \"28810c1c-0d61-437d-b68d-79d6c933120c\") " pod="openshift-monitoring/node-exporter-hkbh6" Jul 24 03:19:24.306443 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.306416 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lg5s\" (UniqueName: \"kubernetes.io/projected/28810c1c-0d61-437d-b68d-79d6c933120c-kube-api-access-7lg5s\") pod \"node-exporter-hkbh6\" (UID: \"28810c1c-0d61-437d-b68d-79d6c933120c\") " pod="openshift-monitoring/node-exporter-hkbh6" Jul 24 03:19:24.307538 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.307506 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vf7nh\" (UniqueName: \"kubernetes.io/projected/3ae74270-00fd-466b-9359-389661481fff-kube-api-access-vf7nh\") pod \"openshift-state-metrics-b8f49f5c6-sgn88\" (UID: \"3ae74270-00fd-466b-9359-389661481fff\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-sgn88" Jul 24 03:19:24.446014 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.445968 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hkbh6" Jul 24 03:19:24.486079 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:19:24.486052 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28810c1c_0d61_437d_b68d_79d6c933120c.slice/crio-b69ec94a53f71484ef92242698ed8f95c4eef395c96a6c8dc07e74ea862a06b7 WatchSource:0}: Error finding container b69ec94a53f71484ef92242698ed8f95c4eef395c96a6c8dc07e74ea862a06b7: Status 404 returned error can't find the container with id b69ec94a53f71484ef92242698ed8f95c4eef395c96a6c8dc07e74ea862a06b7 Jul 24 03:19:24.579471 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.579436 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-797fc79d89-jj4f4"] Jul 24 03:19:24.582364 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:19:24.582323 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda32fb5b0_3f50_486a_9f5b_f989b67cd313.slice/crio-c80fbc09e83680f369dd8ef65e017d8c16c9f854ff8ab5a7624704dcd4c41d2c WatchSource:0}: Error finding container c80fbc09e83680f369dd8ef65e017d8c16c9f854ff8ab5a7624704dcd4c41d2c: Status 404 returned error can't find the container with id c80fbc09e83680f369dd8ef65e017d8c16c9f854ff8ab5a7624704dcd4c41d2c Jul 24 03:19:24.800544 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.800510 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3ae74270-00fd-466b-9359-389661481fff-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-sgn88\" (UID: \"3ae74270-00fd-466b-9359-389661481fff\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-sgn88" Jul 24 03:19:24.802779 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.802753 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3ae74270-00fd-466b-9359-389661481fff-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-sgn88\" (UID: \"3ae74270-00fd-466b-9359-389661481fff\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-sgn88" Jul 24 03:19:24.943214 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.943124 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hkbh6" event={"ID":"28810c1c-0d61-437d-b68d-79d6c933120c","Type":"ContainerStarted","Data":"b69ec94a53f71484ef92242698ed8f95c4eef395c96a6c8dc07e74ea862a06b7"} Jul 24 03:19:24.944501 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:24.944470 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-797fc79d89-jj4f4" event={"ID":"a32fb5b0-3f50-486a-9f5b-f989b67cd313","Type":"ContainerStarted","Data":"c80fbc09e83680f369dd8ef65e017d8c16c9f854ff8ab5a7624704dcd4c41d2c"} Jul 24 03:19:25.012054 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.012021 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-sgn88" Jul 24 03:19:25.062198 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.062156 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 03:19:25.078437 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.078409 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.081183 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.081155 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 03:19:25.081324 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.081185 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 24 03:19:25.081448 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.081425 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 24 03:19:25.081499 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.081458 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 24 03:19:25.081606 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.081588 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 24 03:19:25.081700 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.081615 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 24 03:19:25.081755 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.081620 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 24 03:19:25.081818 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.081801 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 24 03:19:25.081918 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.081881 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 24 03:19:25.082040 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.081982 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 24 03:19:25.082184 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.082163 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-w2cqv\"" Jul 24 03:19:25.203730 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.203638 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.203730 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.203694 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/406d6dd0-7f29-4f7c-9061-086ba77be93d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.203932 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.203771 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-config-volume\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.203932 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.203797 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-web-config\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.203932 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.203843 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.203932 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.203870 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/406d6dd0-7f29-4f7c-9061-086ba77be93d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.203932 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.203908 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.204124 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.203934 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/406d6dd0-7f29-4f7c-9061-086ba77be93d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.204124 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.204013 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsmzl\" (UniqueName: \"kubernetes.io/projected/406d6dd0-7f29-4f7c-9061-086ba77be93d-kube-api-access-hsmzl\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.204124 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.204076 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/406d6dd0-7f29-4f7c-9061-086ba77be93d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.204124 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.204112 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.204321 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.204143 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.204321 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.204163 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/406d6dd0-7f29-4f7c-9061-086ba77be93d-config-out\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.304962 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.304922 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-config-volume\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.305122 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.304977 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-web-config\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.305122 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.305037 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.305122 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.305061 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/406d6dd0-7f29-4f7c-9061-086ba77be93d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.305122 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.305101 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.305324 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.305129 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/406d6dd0-7f29-4f7c-9061-086ba77be93d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.305324 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.305152 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsmzl\" (UniqueName: \"kubernetes.io/projected/406d6dd0-7f29-4f7c-9061-086ba77be93d-kube-api-access-hsmzl\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.305324 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.305192 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/406d6dd0-7f29-4f7c-9061-086ba77be93d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.305324 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.305227 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.305324 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.305263 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.305324 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.305292 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/406d6dd0-7f29-4f7c-9061-086ba77be93d-config-out\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.305625 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.305348 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.305625 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.305373 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/406d6dd0-7f29-4f7c-9061-086ba77be93d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.305795 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.305771 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/406d6dd0-7f29-4f7c-9061-086ba77be93d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.306905 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.306876 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/406d6dd0-7f29-4f7c-9061-086ba77be93d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.308280 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.307964 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/406d6dd0-7f29-4f7c-9061-086ba77be93d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.308280 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.308069 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-config-volume\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.308466 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.308278 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.308643 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.308619 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-web-config\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.308978 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.308956 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/406d6dd0-7f29-4f7c-9061-086ba77be93d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.309197 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.309162 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.309946 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.309924 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/406d6dd0-7f29-4f7c-9061-086ba77be93d-config-out\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.310331 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.310315 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.310582 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.310555 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.311964 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.311943 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.316898 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.316863 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsmzl\" (UniqueName: \"kubernetes.io/projected/406d6dd0-7f29-4f7c-9061-086ba77be93d-kube-api-access-hsmzl\") pod \"alertmanager-main-0\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:25.391015 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:25.390977 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:19:28.231389 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.231346 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bc60e51-e1b5-4f68-a907-600eb7bc0e0b-metrics-certs\") pod \"network-metrics-daemon-gkmfm\" (UID: \"1bc60e51-e1b5-4f68-a907-600eb7bc0e0b\") " pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:19:28.231815 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.231410 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/041beb02-32e9-4b8e-8f9d-34ee8d1a72c6-original-pull-secret\") pod \"global-pull-secret-syncer-8hlhr\" (UID: \"041beb02-32e9-4b8e-8f9d-34ee8d1a72c6\") " pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:19:28.233614 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.233562 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 24 03:19:28.233614 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.233589 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 24 03:19:28.244715 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.244689 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bc60e51-e1b5-4f68-a907-600eb7bc0e0b-metrics-certs\") pod \"network-metrics-daemon-gkmfm\" (UID: \"1bc60e51-e1b5-4f68-a907-600eb7bc0e0b\") " pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:19:28.244972 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.244953 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/041beb02-32e9-4b8e-8f9d-34ee8d1a72c6-original-pull-secret\") pod \"global-pull-secret-syncer-8hlhr\" (UID: \"041beb02-32e9-4b8e-8f9d-34ee8d1a72c6\") " pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:19:28.254107 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.254085 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ph9hg\"" Jul 24 03:19:28.262700 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.262678 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkmfm" Jul 24 03:19:28.288823 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.288788 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-d4b79fc88-njng4"] Jul 24 03:19:28.303335 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.303311 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-d4b79fc88-njng4"] Jul 24 03:19:28.303473 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.303367 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-d4b79fc88-njng4" Jul 24 03:19:28.305646 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.305613 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-xd5xz\"" Jul 24 03:19:28.305646 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.305635 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 24 03:19:28.305898 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.305715 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-da6gtdjv2sfjc\"" Jul 24 03:19:28.306457 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.306433 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 24 03:19:28.306620 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.306602 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 24 03:19:28.306710 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.306653 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 24 03:19:28.332656 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.332622 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfqtr\" (UniqueName: \"kubernetes.io/projected/1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85-kube-api-access-qfqtr\") pod \"network-check-target-52h8l\" (UID: \"1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85\") " pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:19:28.334841 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.334823 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 24 03:19:28.345946 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.345922 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 24 03:19:28.356605 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.356579 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfqtr\" (UniqueName: \"kubernetes.io/projected/1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85-kube-api-access-qfqtr\") pod \"network-check-target-52h8l\" (UID: \"1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85\") " pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:19:28.433856 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.433823 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b3f0b141-5296-41c1-afd2-649194830051-audit-log\") pod \"metrics-server-d4b79fc88-njng4\" (UID: \"b3f0b141-5296-41c1-afd2-649194830051\") " pod="openshift-monitoring/metrics-server-d4b79fc88-njng4" Jul 24 03:19:28.433856 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.433859 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b3f0b141-5296-41c1-afd2-649194830051-client-ca-bundle\") pod \"metrics-server-d4b79fc88-njng4\" (UID: \"b3f0b141-5296-41c1-afd2-649194830051\") " pod="openshift-monitoring/metrics-server-d4b79fc88-njng4" Jul 24 03:19:28.434090 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.433894 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b3f0b141-5296-41c1-afd2-649194830051-metrics-server-audit-profiles\") pod \"metrics-server-d4b79fc88-njng4\" (UID: \"b3f0b141-5296-41c1-afd2-649194830051\") " pod="openshift-monitoring/metrics-server-d4b79fc88-njng4" Jul 24 03:19:28.434090 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.433946 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b3f0b141-5296-41c1-afd2-649194830051-secret-metrics-server-tls\") pod \"metrics-server-d4b79fc88-njng4\" (UID: \"b3f0b141-5296-41c1-afd2-649194830051\") " pod="openshift-monitoring/metrics-server-d4b79fc88-njng4" Jul 24 03:19:28.434090 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.433980 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95fnr\" (UniqueName: \"kubernetes.io/projected/b3f0b141-5296-41c1-afd2-649194830051-kube-api-access-95fnr\") pod \"metrics-server-d4b79fc88-njng4\" (UID: \"b3f0b141-5296-41c1-afd2-649194830051\") " pod="openshift-monitoring/metrics-server-d4b79fc88-njng4" Jul 24 03:19:28.434090 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.434053 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b3f0b141-5296-41c1-afd2-649194830051-secret-metrics-server-client-certs\") pod \"metrics-server-d4b79fc88-njng4\" (UID: \"b3f0b141-5296-41c1-afd2-649194830051\") " pod="openshift-monitoring/metrics-server-d4b79fc88-njng4" Jul 24 03:19:28.434256 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.434112 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3f0b141-5296-41c1-afd2-649194830051-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-d4b79fc88-njng4\" (UID: \"b3f0b141-5296-41c1-afd2-649194830051\") " pod="openshift-monitoring/metrics-server-d4b79fc88-njng4" Jul 24 03:19:28.534632 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.534526 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b3f0b141-5296-41c1-afd2-649194830051-secret-metrics-server-tls\") pod \"metrics-server-d4b79fc88-njng4\" (UID: \"b3f0b141-5296-41c1-afd2-649194830051\") " pod="openshift-monitoring/metrics-server-d4b79fc88-njng4" Jul 24 03:19:28.534632 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.534584 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95fnr\" (UniqueName: \"kubernetes.io/projected/b3f0b141-5296-41c1-afd2-649194830051-kube-api-access-95fnr\") pod \"metrics-server-d4b79fc88-njng4\" (UID: \"b3f0b141-5296-41c1-afd2-649194830051\") " pod="openshift-monitoring/metrics-server-d4b79fc88-njng4" Jul 24 03:19:28.534632 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.534618 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b3f0b141-5296-41c1-afd2-649194830051-secret-metrics-server-client-certs\") pod \"metrics-server-d4b79fc88-njng4\" (UID: \"b3f0b141-5296-41c1-afd2-649194830051\") " pod="openshift-monitoring/metrics-server-d4b79fc88-njng4" Jul 24 03:19:28.534930 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.534658 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3f0b141-5296-41c1-afd2-649194830051-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-d4b79fc88-njng4\" (UID: \"b3f0b141-5296-41c1-afd2-649194830051\") " pod="openshift-monitoring/metrics-server-d4b79fc88-njng4" Jul 24 03:19:28.534930 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.534716 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b3f0b141-5296-41c1-afd2-649194830051-audit-log\") pod \"metrics-server-d4b79fc88-njng4\" (UID: \"b3f0b141-5296-41c1-afd2-649194830051\") " pod="openshift-monitoring/metrics-server-d4b79fc88-njng4" Jul 24 03:19:28.534930 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.534739 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b3f0b141-5296-41c1-afd2-649194830051-client-ca-bundle\") pod \"metrics-server-d4b79fc88-njng4\" (UID: \"b3f0b141-5296-41c1-afd2-649194830051\") " pod="openshift-monitoring/metrics-server-d4b79fc88-njng4" Jul 24 03:19:28.534930 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.534771 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b3f0b141-5296-41c1-afd2-649194830051-metrics-server-audit-profiles\") pod \"metrics-server-d4b79fc88-njng4\" (UID: \"b3f0b141-5296-41c1-afd2-649194830051\") " pod="openshift-monitoring/metrics-server-d4b79fc88-njng4" Jul 24 03:19:28.535287 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.535195 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b3f0b141-5296-41c1-afd2-649194830051-audit-log\") pod \"metrics-server-d4b79fc88-njng4\" (UID: \"b3f0b141-5296-41c1-afd2-649194830051\") " pod="openshift-monitoring/metrics-server-d4b79fc88-njng4" Jul 24 03:19:28.535836 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.535781 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3f0b141-5296-41c1-afd2-649194830051-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-d4b79fc88-njng4\" (UID: \"b3f0b141-5296-41c1-afd2-649194830051\") " pod="openshift-monitoring/metrics-server-d4b79fc88-njng4" Jul 24 03:19:28.535929 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.535911 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b3f0b141-5296-41c1-afd2-649194830051-metrics-server-audit-profiles\") pod \"metrics-server-d4b79fc88-njng4\" (UID: \"b3f0b141-5296-41c1-afd2-649194830051\") " pod="openshift-monitoring/metrics-server-d4b79fc88-njng4" Jul 24 03:19:28.536551 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.536525 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-w5g4d\"" Jul 24 03:19:28.537413 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.537359 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b3f0b141-5296-41c1-afd2-649194830051-secret-metrics-server-client-certs\") pod \"metrics-server-d4b79fc88-njng4\" (UID: \"b3f0b141-5296-41c1-afd2-649194830051\") " pod="openshift-monitoring/metrics-server-d4b79fc88-njng4" Jul 24 03:19:28.537498 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.537449 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b3f0b141-5296-41c1-afd2-649194830051-secret-metrics-server-tls\") pod \"metrics-server-d4b79fc88-njng4\" (UID: \"b3f0b141-5296-41c1-afd2-649194830051\") " pod="openshift-monitoring/metrics-server-d4b79fc88-njng4" Jul 24 03:19:28.537498 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.537481 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b3f0b141-5296-41c1-afd2-649194830051-client-ca-bundle\") pod \"metrics-server-d4b79fc88-njng4\" (UID: \"b3f0b141-5296-41c1-afd2-649194830051\") " pod="openshift-monitoring/metrics-server-d4b79fc88-njng4" Jul 24 03:19:28.543222 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.543202 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95fnr\" (UniqueName: \"kubernetes.io/projected/b3f0b141-5296-41c1-afd2-649194830051-kube-api-access-95fnr\") pod \"metrics-server-d4b79fc88-njng4\" (UID: \"b3f0b141-5296-41c1-afd2-649194830051\") " pod="openshift-monitoring/metrics-server-d4b79fc88-njng4" Jul 24 03:19:28.543322 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.543297 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hlhr" Jul 24 03:19:28.545045 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.545022 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:19:28.615079 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.615057 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-d4b79fc88-njng4" Jul 24 03:19:28.783419 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.783383 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-797fc79d89-jj4f4"] Jul 24 03:19:28.851115 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.851088 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-57ddd6f59f-7nt49"] Jul 24 03:19:28.866154 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.866121 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57ddd6f59f-7nt49" Jul 24 03:19:28.867856 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.867820 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57ddd6f59f-7nt49"] Jul 24 03:19:28.874062 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.874033 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 24 03:19:28.938293 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.938261 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f61b295-42d3-45a4-9437-a6ec0e961459-trusted-ca-bundle\") pod \"console-57ddd6f59f-7nt49\" (UID: \"6f61b295-42d3-45a4-9437-a6ec0e961459\") " pod="openshift-console/console-57ddd6f59f-7nt49" Jul 24 03:19:28.938480 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.938302 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f61b295-42d3-45a4-9437-a6ec0e961459-console-serving-cert\") pod \"console-57ddd6f59f-7nt49\" (UID: \"6f61b295-42d3-45a4-9437-a6ec0e961459\") " pod="openshift-console/console-57ddd6f59f-7nt49" Jul 24 03:19:28.938480 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.938317 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f61b295-42d3-45a4-9437-a6ec0e961459-console-oauth-config\") pod \"console-57ddd6f59f-7nt49\" (UID: \"6f61b295-42d3-45a4-9437-a6ec0e961459\") " pod="openshift-console/console-57ddd6f59f-7nt49" Jul 24 03:19:28.938480 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.938344 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f61b295-42d3-45a4-9437-a6ec0e961459-service-ca\") pod \"console-57ddd6f59f-7nt49\" (UID: \"6f61b295-42d3-45a4-9437-a6ec0e961459\") " pod="openshift-console/console-57ddd6f59f-7nt49" Jul 24 03:19:28.938480 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.938368 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f61b295-42d3-45a4-9437-a6ec0e961459-oauth-serving-cert\") pod \"console-57ddd6f59f-7nt49\" (UID: \"6f61b295-42d3-45a4-9437-a6ec0e961459\") " pod="openshift-console/console-57ddd6f59f-7nt49" Jul 24 03:19:28.938480 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.938449 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dszfd\" (UniqueName: \"kubernetes.io/projected/6f61b295-42d3-45a4-9437-a6ec0e961459-kube-api-access-dszfd\") pod \"console-57ddd6f59f-7nt49\" (UID: \"6f61b295-42d3-45a4-9437-a6ec0e961459\") " pod="openshift-console/console-57ddd6f59f-7nt49" Jul 24 03:19:28.938714 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:28.938533 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f61b295-42d3-45a4-9437-a6ec0e961459-console-config\") pod \"console-57ddd6f59f-7nt49\" (UID: \"6f61b295-42d3-45a4-9437-a6ec0e961459\") " pod="openshift-console/console-57ddd6f59f-7nt49" Jul 24 03:19:29.039057 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:29.039019 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f61b295-42d3-45a4-9437-a6ec0e961459-trusted-ca-bundle\") pod \"console-57ddd6f59f-7nt49\" (UID: \"6f61b295-42d3-45a4-9437-a6ec0e961459\") " pod="openshift-console/console-57ddd6f59f-7nt49" Jul 24 03:19:29.039308 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:29.039075 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f61b295-42d3-45a4-9437-a6ec0e961459-console-serving-cert\") pod \"console-57ddd6f59f-7nt49\" (UID: \"6f61b295-42d3-45a4-9437-a6ec0e961459\") " pod="openshift-console/console-57ddd6f59f-7nt49" Jul 24 03:19:29.039308 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:29.039099 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f61b295-42d3-45a4-9437-a6ec0e961459-console-oauth-config\") pod \"console-57ddd6f59f-7nt49\" (UID: \"6f61b295-42d3-45a4-9437-a6ec0e961459\") " pod="openshift-console/console-57ddd6f59f-7nt49" Jul 24 03:19:29.039308 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:29.039127 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f61b295-42d3-45a4-9437-a6ec0e961459-service-ca\") pod \"console-57ddd6f59f-7nt49\" (UID: \"6f61b295-42d3-45a4-9437-a6ec0e961459\") " pod="openshift-console/console-57ddd6f59f-7nt49" Jul 24 03:19:29.039308 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:29.039148 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f61b295-42d3-45a4-9437-a6ec0e961459-oauth-serving-cert\") pod \"console-57ddd6f59f-7nt49\" (UID: \"6f61b295-42d3-45a4-9437-a6ec0e961459\") " pod="openshift-console/console-57ddd6f59f-7nt49" Jul 24 03:19:29.039308 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:29.039170 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dszfd\" (UniqueName: \"kubernetes.io/projected/6f61b295-42d3-45a4-9437-a6ec0e961459-kube-api-access-dszfd\") pod \"console-57ddd6f59f-7nt49\" (UID: \"6f61b295-42d3-45a4-9437-a6ec0e961459\") " pod="openshift-console/console-57ddd6f59f-7nt49" Jul 24 03:19:29.039308 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:29.039234 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f61b295-42d3-45a4-9437-a6ec0e961459-console-config\") pod \"console-57ddd6f59f-7nt49\" (UID: \"6f61b295-42d3-45a4-9437-a6ec0e961459\") " pod="openshift-console/console-57ddd6f59f-7nt49" Jul 24 03:19:29.040097 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:29.040042 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f61b295-42d3-45a4-9437-a6ec0e961459-service-ca\") pod \"console-57ddd6f59f-7nt49\" (UID: \"6f61b295-42d3-45a4-9437-a6ec0e961459\") " pod="openshift-console/console-57ddd6f59f-7nt49" Jul 24 03:19:29.040097 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:29.040066 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f61b295-42d3-45a4-9437-a6ec0e961459-trusted-ca-bundle\") pod \"console-57ddd6f59f-7nt49\" (UID: \"6f61b295-42d3-45a4-9437-a6ec0e961459\") " pod="openshift-console/console-57ddd6f59f-7nt49" Jul 24 03:19:29.040097 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:29.040072 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f61b295-42d3-45a4-9437-a6ec0e961459-oauth-serving-cert\") pod \"console-57ddd6f59f-7nt49\" (UID: \"6f61b295-42d3-45a4-9437-a6ec0e961459\") " pod="openshift-console/console-57ddd6f59f-7nt49" Jul 24 03:19:29.040329 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:29.040134 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f61b295-42d3-45a4-9437-a6ec0e961459-console-config\") pod \"console-57ddd6f59f-7nt49\" (UID: \"6f61b295-42d3-45a4-9437-a6ec0e961459\") " pod="openshift-console/console-57ddd6f59f-7nt49" Jul 24 03:19:29.042005 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:29.041972 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f61b295-42d3-45a4-9437-a6ec0e961459-console-oauth-config\") pod \"console-57ddd6f59f-7nt49\" (UID: \"6f61b295-42d3-45a4-9437-a6ec0e961459\") " pod="openshift-console/console-57ddd6f59f-7nt49" Jul 24 03:19:29.042189 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:29.042168 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f61b295-42d3-45a4-9437-a6ec0e961459-console-serving-cert\") pod \"console-57ddd6f59f-7nt49\" (UID: \"6f61b295-42d3-45a4-9437-a6ec0e961459\") " pod="openshift-console/console-57ddd6f59f-7nt49" Jul 24 03:19:29.047598 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:29.047576 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dszfd\" (UniqueName: \"kubernetes.io/projected/6f61b295-42d3-45a4-9437-a6ec0e961459-kube-api-access-dszfd\") pod \"console-57ddd6f59f-7nt49\" (UID: \"6f61b295-42d3-45a4-9437-a6ec0e961459\") " pod="openshift-console/console-57ddd6f59f-7nt49" Jul 24 03:19:29.178786 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:29.178703 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57ddd6f59f-7nt49" Jul 24 03:19:30.168729 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.168694 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 24 03:19:30.191122 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.191047 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 24 03:19:30.191288 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.191193 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.194257 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.194230 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 24 03:19:30.194799 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.194438 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 24 03:19:30.194799 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.194645 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 24 03:19:30.194799 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.194687 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6rqb5c3grvcau\"" Jul 24 03:19:30.194799 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.194777 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-ztqd4\"" Jul 24 03:19:30.195330 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.194971 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 24 03:19:30.195407 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.195340 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 24 03:19:30.195493 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.195478 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 24 03:19:30.195608 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.195587 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 24 03:19:30.195690 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.195619 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 24 03:19:30.195690 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.195652 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 24 03:19:30.195880 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.195862 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 24 03:19:30.196099 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.196034 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 24 03:19:30.196269 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.196249 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 24 03:19:30.197493 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.197473 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 24 03:19:30.201515 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.200817 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 24 03:19:30.249276 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.249233 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.249471 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.249283 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.249471 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.249318 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-web-config\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.249471 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.249410 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.249471 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.249448 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.249722 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.249477 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.249722 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.249501 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.249722 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.249546 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdp7x\" (UniqueName: \"kubernetes.io/projected/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-kube-api-access-rdp7x\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.249722 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.249580 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.249722 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.249607 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.249722 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.249644 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.249722 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.249723 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-config\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.250120 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.249745 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.250120 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.249762 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.250120 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.249787 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.250120 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.249808 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-config-out\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.250120 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.249824 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.250120 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.249851 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.250120 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.249874 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.250120 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.249907 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.352461 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.352409 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.352461 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.352468 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.352741 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.352501 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.352741 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.352532 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-web-config\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.352741 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.352571 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.352741 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.352597 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.352741 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.352628 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.352741 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.352652 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.352741 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.352696 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdp7x\" (UniqueName: \"kubernetes.io/projected/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-kube-api-access-rdp7x\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.352741 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.352736 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.353140 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.352762 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.353140 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.352791 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.353140 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.352826 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-config\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.353140 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.352840 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.353140 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.352859 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.353140 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.352879 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.353140 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.352898 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-config-out\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.353140 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.352912 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.353140 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.352934 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.353140 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.352949 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.354077 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.353768 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.354493 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.354170 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.354493 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.354409 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.354970 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.354943 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.355521 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.355496 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.356642 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.356615 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.356825 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.356803 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.356962 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.356942 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.357341 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.357239 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.357856 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.357832 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.358223 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.358203 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.359136 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.359116 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.359273 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.359217 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.359273 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.359243 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-config\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.359413 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.359393 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.360028 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.360009 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.360149 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.360130 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-web-config\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.360215 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.360191 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-config-out\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.361631 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.361593 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.363602 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.363573 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdp7x\" (UniqueName: \"kubernetes.io/projected/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-kube-api-access-rdp7x\") pod \"prometheus-k8s-0\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.505577 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.505481 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:19:30.840773 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:30.840745 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ffrlv" Jul 24 03:19:31.936925 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:31.936893 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-rbhtr" Jul 24 03:19:32.425215 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:32.425177 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-52h8l"] Jul 24 03:19:32.442718 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:32.441691 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 24 03:19:32.462296 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:32.462227 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57ddd6f59f-7nt49"] Jul 24 03:19:32.465774 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:32.465709 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-d4b79fc88-njng4"] Jul 24 03:19:32.501169 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:32.501125 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 03:19:32.504875 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:19:32.504839 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fe8d46d_5e0b_4b01_a7fb_c86dcbddcf85.slice/crio-29af0dfefb2923bbaddc0e53d805480466d82569fe1ed9e32a78d7433dbbb3e1 WatchSource:0}: Error finding container 29af0dfefb2923bbaddc0e53d805480466d82569fe1ed9e32a78d7433dbbb3e1: Status 404 returned error can't find the container with id 29af0dfefb2923bbaddc0e53d805480466d82569fe1ed9e32a78d7433dbbb3e1 Jul 24 03:19:32.507242 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:19:32.507170 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode523c03c_b9eb_44cb_b2fd_967c7b02f14f.slice/crio-5275b15824f50dbcbbe70f7b36b08490d778c035fa7c7cd5a687a619a50dbe77 WatchSource:0}: Error finding container 5275b15824f50dbcbbe70f7b36b08490d778c035fa7c7cd5a687a619a50dbe77: Status 404 returned error can't find the container with id 5275b15824f50dbcbbe70f7b36b08490d778c035fa7c7cd5a687a619a50dbe77 Jul 24 03:19:32.507680 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:19:32.507503 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f61b295_42d3_45a4_9437_a6ec0e961459.slice/crio-a2ed95b181328d6ee1d2f14d3b6f3d382eb65ea6ed261fcdd46c14bbeef41334 WatchSource:0}: Error finding container a2ed95b181328d6ee1d2f14d3b6f3d382eb65ea6ed261fcdd46c14bbeef41334: Status 404 returned error can't find the container with id a2ed95b181328d6ee1d2f14d3b6f3d382eb65ea6ed261fcdd46c14bbeef41334 Jul 24 03:19:32.508597 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:19:32.508579 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3f0b141_5296_41c1_afd2_649194830051.slice/crio-8b59a6f1215d8f0a4aa8bee4507c72898ad224ee887919c0cec63023f2848867 WatchSource:0}: Error finding container 8b59a6f1215d8f0a4aa8bee4507c72898ad224ee887919c0cec63023f2848867: Status 404 returned error can't find the container with id 8b59a6f1215d8f0a4aa8bee4507c72898ad224ee887919c0cec63023f2848867 Jul 24 03:19:32.510077 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:19:32.510056 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod406d6dd0_7f29_4f7c_9061_086ba77be93d.slice/crio-a3a712c147ac63fb4c3e454c2287bb446f3d35d406b9e9f5d5ed5b096858cbdd WatchSource:0}: Error finding container a3a712c147ac63fb4c3e454c2287bb446f3d35d406b9e9f5d5ed5b096858cbdd: Status 404 returned error can't find the container with id a3a712c147ac63fb4c3e454c2287bb446f3d35d406b9e9f5d5ed5b096858cbdd Jul 24 03:19:32.727476 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:32.727404 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8hlhr"] Jul 24 03:19:32.728298 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:32.728225 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-sgn88"] Jul 24 03:19:32.729449 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:32.729427 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gkmfm"] Jul 24 03:19:32.731600 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:19:32.731575 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod041beb02_32e9_4b8e_8f9d_34ee8d1a72c6.slice/crio-67ca5e3d04d0c0a2e643a4b5970d163009e3cabd646c911b755ff9b301033644 WatchSource:0}: Error finding container 67ca5e3d04d0c0a2e643a4b5970d163009e3cabd646c911b755ff9b301033644: Status 404 returned error can't find the container with id 67ca5e3d04d0c0a2e643a4b5970d163009e3cabd646c911b755ff9b301033644 Jul 24 03:19:32.735193 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:19:32.734716 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ae74270_00fd_466b_9359_389661481fff.slice/crio-034a023844d91bb3e55d7557c61f1cd555bb043b94ff0eaa1746e93d872cccc8 WatchSource:0}: Error finding container 034a023844d91bb3e55d7557c61f1cd555bb043b94ff0eaa1746e93d872cccc8: Status 404 returned error can't find the container with id 034a023844d91bb3e55d7557c61f1cd555bb043b94ff0eaa1746e93d872cccc8 Jul 24 03:19:32.738533 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:19:32.738493 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bc60e51_e1b5_4f68_a907_600eb7bc0e0b.slice/crio-9b8399713e979c0a8ddb389776dba4f321b35be52753e5671e1899f5e9b324af WatchSource:0}: Error finding container 9b8399713e979c0a8ddb389776dba4f321b35be52753e5671e1899f5e9b324af: Status 404 returned error can't find the container with id 9b8399713e979c0a8ddb389776dba4f321b35be52753e5671e1899f5e9b324af Jul 24 03:19:32.985743 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:32.983637 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8hlhr" event={"ID":"041beb02-32e9-4b8e-8f9d-34ee8d1a72c6","Type":"ContainerStarted","Data":"67ca5e3d04d0c0a2e643a4b5970d163009e3cabd646c911b755ff9b301033644"} Jul 24 03:19:32.988354 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:32.988258 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57ddd6f59f-7nt49" event={"ID":"6f61b295-42d3-45a4-9437-a6ec0e961459","Type":"ContainerStarted","Data":"a2ed95b181328d6ee1d2f14d3b6f3d382eb65ea6ed261fcdd46c14bbeef41334"} Jul 24 03:19:32.990698 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:32.990575 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"406d6dd0-7f29-4f7c-9061-086ba77be93d","Type":"ContainerStarted","Data":"a3a712c147ac63fb4c3e454c2287bb446f3d35d406b9e9f5d5ed5b096858cbdd"} Jul 24 03:19:32.995717 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:32.995549 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-sgn88" event={"ID":"3ae74270-00fd-466b-9359-389661481fff","Type":"ContainerStarted","Data":"8c18d77119fe90c47c5b0b4c2bb44929093aa7ecd49d2354e7d77b6e8bde08e4"} Jul 24 03:19:32.995717 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:32.995607 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-sgn88" event={"ID":"3ae74270-00fd-466b-9359-389661481fff","Type":"ContainerStarted","Data":"e2aa1c0e12d7c778022c61487d80d6e8eee3ca98fbfeeae904d4d7ddbc880393"} Jul 24 03:19:32.995717 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:32.995638 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-sgn88" event={"ID":"3ae74270-00fd-466b-9359-389661481fff","Type":"ContainerStarted","Data":"034a023844d91bb3e55d7557c61f1cd555bb043b94ff0eaa1746e93d872cccc8"} Jul 24 03:19:32.998210 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:32.998127 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-d4b79fc88-njng4" event={"ID":"b3f0b141-5296-41c1-afd2-649194830051","Type":"ContainerStarted","Data":"8b59a6f1215d8f0a4aa8bee4507c72898ad224ee887919c0cec63023f2848867"} Jul 24 03:19:33.002809 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:33.001839 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-52h8l" event={"ID":"1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85","Type":"ContainerStarted","Data":"29af0dfefb2923bbaddc0e53d805480466d82569fe1ed9e32a78d7433dbbb3e1"} Jul 24 03:19:33.007471 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:33.006721 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-9h4l2" event={"ID":"18fbf1b4-55a4-4e8c-806e-408fb712a12a","Type":"ContainerStarted","Data":"319e2b946bc0a90b89b21a61b7d509b8472f87a71cd3b46ad0431e953ec0e0dd"} Jul 24 03:19:33.007471 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:33.007196 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-9h4l2" Jul 24 03:19:33.012407 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:33.012039 2579 patch_prober.go:28] interesting pod/downloads-7c97dc4555-9h4l2 container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.134.0.10:8080/\": dial tcp 10.134.0.10:8080: connect: connection refused" start-of-body= Jul 24 03:19:33.012407 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:33.012100 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-7c97dc4555-9h4l2" podUID="18fbf1b4-55a4-4e8c-806e-408fb712a12a" containerName="download-server" probeResult="failure" output="Get \"http://10.134.0.10:8080/\": dial tcp 10.134.0.10:8080: connect: connection refused" Jul 24 03:19:33.025791 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:33.025762 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e523c03c-b9eb-44cb-b2fd-967c7b02f14f","Type":"ContainerStarted","Data":"5275b15824f50dbcbbe70f7b36b08490d778c035fa7c7cd5a687a619a50dbe77"} Jul 24 03:19:33.031961 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:33.031481 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-9h4l2" podStartSLOduration=1.785749558 podStartE2EDuration="21.031465099s" totalImagesPullingTime="19.245715541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:19:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:19:33.03031106 +0000 UTC m=+70.305882411" watchObservedRunningTime="2026-07-24 03:19:33.031465099 +0000 UTC m=+70.307036450" Jul 24 03:19:33.036938 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:33.036617 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5t6sr" event={"ID":"cf923b24-c681-4393-b0a9-7db5c0183552","Type":"ContainerStarted","Data":"6d10641890e17765b4906421244982779a6023ee704fca4cd38204953f60304e"} Jul 24 03:19:33.041318 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:33.041262 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gkmfm" event={"ID":"1bc60e51-e1b5-4f68-a907-600eb7bc0e0b","Type":"ContainerStarted","Data":"9b8399713e979c0a8ddb389776dba4f321b35be52753e5671e1899f5e9b324af"} Jul 24 03:19:34.060258 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:34.060017 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-9h4l2" Jul 24 03:19:48.144577 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:48.144534 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gkmfm" event={"ID":"1bc60e51-e1b5-4f68-a907-600eb7bc0e0b","Type":"ContainerStarted","Data":"a1f5440e3a7847052a73c8275d96027f7403dc799b9e3e6abcfbcd2e79dbfd2c"} Jul 24 03:19:48.145173 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:48.144648 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gkmfm" event={"ID":"1bc60e51-e1b5-4f68-a907-600eb7bc0e0b","Type":"ContainerStarted","Data":"2a875fc1902e8ede6d48d2b8a3a761fd0a3e9e2715b659610c92027819f20a39"} Jul 24 03:19:48.148927 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:48.148897 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8hlhr" event={"ID":"041beb02-32e9-4b8e-8f9d-34ee8d1a72c6","Type":"ContainerStarted","Data":"ab232877a4c9c735e210e1f430f94a458443a7227003c467b9f19a6b70c512a8"} Jul 24 03:19:48.150245 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:48.150220 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57ddd6f59f-7nt49" event={"ID":"6f61b295-42d3-45a4-9437-a6ec0e961459","Type":"ContainerStarted","Data":"1a39958685134a59689d266501274902dfc4933543064b1c7499ac5b5b6b410a"} Jul 24 03:19:48.151588 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:48.151568 2579 generic.go:356] "Generic (PLEG): container finished" podID="406d6dd0-7f29-4f7c-9061-086ba77be93d" containerID="6519e86ed1dcf01b70457703b799e0ef66d3774910f3c7498ac78735a15e105a" exitCode=0 Jul 24 03:19:48.151721 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:48.151656 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"406d6dd0-7f29-4f7c-9061-086ba77be93d","Type":"ContainerDied","Data":"6519e86ed1dcf01b70457703b799e0ef66d3774910f3c7498ac78735a15e105a"} Jul 24 03:19:48.153654 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:48.153629 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-sgn88" event={"ID":"3ae74270-00fd-466b-9359-389661481fff","Type":"ContainerStarted","Data":"a97fe8326f7e41719b30b84cb1d65b9b94922071af4b80e55000ab8a3dbb69dd"} Jul 24 03:19:48.155099 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:48.155077 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-d4b79fc88-njng4" event={"ID":"b3f0b141-5296-41c1-afd2-649194830051","Type":"ContainerStarted","Data":"25e1dddee08f201368d8e37df2acd8a8afe664b4feebfa9edd334925959c297a"} Jul 24 03:19:48.156474 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:48.156450 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-52h8l" event={"ID":"1fe8d46d-5e0b-4b01-a7fb-c86dcbddcf85","Type":"ContainerStarted","Data":"437431bebcdda0e49a7bd9d034eb75ad5295784b50b054684e5885ee12e75aed"} Jul 24 03:19:48.156574 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:48.156509 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:19:48.157799 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:48.157780 2579 generic.go:356] "Generic (PLEG): container finished" podID="e523c03c-b9eb-44cb-b2fd-967c7b02f14f" containerID="e9b9884d2041a304c3bb841c27eec6b14e2a825ef4d402af2d2adb66a264fa9d" exitCode=0 Jul 24 03:19:48.157905 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:48.157848 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e523c03c-b9eb-44cb-b2fd-967c7b02f14f","Type":"ContainerDied","Data":"e9b9884d2041a304c3bb841c27eec6b14e2a825ef4d402af2d2adb66a264fa9d"} Jul 24 03:19:48.160308 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:48.159510 2579 generic.go:356] "Generic (PLEG): container finished" podID="28810c1c-0d61-437d-b68d-79d6c933120c" containerID="0d1e2be4c42d926d1809212e4b134d4fd3cdc125f1edaa5a960763a2b995627f" exitCode=0 Jul 24 03:19:48.160308 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:48.159601 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hkbh6" event={"ID":"28810c1c-0d61-437d-b68d-79d6c933120c","Type":"ContainerDied","Data":"0d1e2be4c42d926d1809212e4b134d4fd3cdc125f1edaa5a960763a2b995627f"} Jul 24 03:19:48.161883 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:48.161860 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-797fc79d89-jj4f4" event={"ID":"a32fb5b0-3f50-486a-9f5b-f989b67cd313","Type":"ContainerStarted","Data":"bc75cb99e8d5d28b67c182385d48dd3df4d9559069270831209c1ade46d863cd"} Jul 24 03:19:48.164206 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:48.164163 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-gkmfm" podStartSLOduration=69.659873066 podStartE2EDuration="1m24.164148227s" totalImagesPullingTime="14.504275161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:18:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:19:48.163182085 +0000 UTC m=+85.438753435" watchObservedRunningTime="2026-07-24 03:19:48.164148227 +0000 UTC m=+85.439719579" Jul 24 03:19:48.164320 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:48.164290 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5t6sr" event={"ID":"cf923b24-c681-4393-b0a9-7db5c0183552","Type":"ContainerStarted","Data":"cd0f37887eb4eec974ac047f6039cc4af2d8dc9f9fbc2de43b5558f32a30dacc"} Jul 24 03:19:48.251544 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:48.251501 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-d4b79fc88-njng4" podStartSLOduration=5.567716399 podStartE2EDuration="20.251485433s" totalImagesPullingTime="14.683769034s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:19:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:19:48.250650025 +0000 UTC m=+85.526221373" watchObservedRunningTime="2026-07-24 03:19:48.251485433 +0000 UTC m=+85.527056782" Jul 24 03:19:48.268760 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:48.268639 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-797fc79d89-jj4f4" podStartSLOduration=4.6420473300000005 podStartE2EDuration="27.268621719s" totalImagesPullingTime="22.626574389s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:19:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:19:48.268094457 +0000 UTC m=+85.543665811" watchObservedRunningTime="2026-07-24 03:19:48.268621719 +0000 UTC m=+85.544193069" Jul 24 03:19:48.288778 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:48.288736 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-sgn88" podStartSLOduration=10.992518895 podStartE2EDuration="25.288720086s" totalImagesPullingTime="14.296201191s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:19:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:19:48.286963732 +0000 UTC m=+85.562535082" watchObservedRunningTime="2026-07-24 03:19:48.288720086 +0000 UTC m=+85.564291436" Jul 24 03:19:48.305161 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:48.305114 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-57ddd6f59f-7nt49" podStartSLOduration=5.590812524 podStartE2EDuration="20.305094755s" totalImagesPullingTime="14.714282231s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:19:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:19:48.303486318 +0000 UTC m=+85.579057669" watchObservedRunningTime="2026-07-24 03:19:48.305094755 +0000 UTC m=+85.580666105" Jul 24 03:19:48.331704 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:48.331642 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-52h8l" podStartSLOduration=69.620998562 podStartE2EDuration="1m24.331627619s" totalImagesPullingTime="14.710629057s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:18:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:19:48.331104469 +0000 UTC m=+85.606675819" watchObservedRunningTime="2026-07-24 03:19:48.331627619 +0000 UTC m=+85.607198970" Jul 24 03:19:48.332770 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:48.332728 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8hlhr" podStartSLOduration=69.804977918 podStartE2EDuration="1m24.332718223s" totalImagesPullingTime="14.527740305s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 03:18:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:19:48.318241849 +0000 UTC m=+85.593813196" watchObservedRunningTime="2026-07-24 03:19:48.332718223 +0000 UTC m=+85.608289572" Jul 24 03:19:48.350570 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:48.350510 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-5t6sr" podStartSLOduration=2.6102010780000002 podStartE2EDuration="36.350481047s" totalImagesPullingTime="33.740279969s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 03:19:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:19:48.350485107 +0000 UTC m=+85.626056450" watchObservedRunningTime="2026-07-24 03:19:48.350481047 +0000 UTC m=+85.626052399" Jul 24 03:19:48.615966 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:48.615933 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-d4b79fc88-njng4" Jul 24 03:19:48.616152 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:48.615994 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-d4b79fc88-njng4" Jul 24 03:19:49.179429 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:49.179022 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-57ddd6f59f-7nt49" Jul 24 03:19:49.179429 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:49.179248 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-57ddd6f59f-7nt49" Jul 24 03:19:49.180408 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:49.180201 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hkbh6" event={"ID":"28810c1c-0d61-437d-b68d-79d6c933120c","Type":"ContainerStarted","Data":"90f8a61b9d999b62b6a8c8319d77d841f817311f2944e3433e7717cfabf567f3"} Jul 24 03:19:49.180408 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:49.180259 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hkbh6" event={"ID":"28810c1c-0d61-437d-b68d-79d6c933120c","Type":"ContainerStarted","Data":"be650b6b82b11d92543fbfb999fe7f88e8fb67cd753533d3a8d6bfa5f652d790"} Jul 24 03:19:49.187340 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:49.187320 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-57ddd6f59f-7nt49" Jul 24 03:19:49.201828 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:49.201404 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-hkbh6" podStartSLOduration=8.404480294 podStartE2EDuration="26.20138532s" totalImagesPullingTime="17.661052276s" totalInitContainerRuntime="135.85275ms" isStatefulPod=true podCreationTimestamp="2026-07-24 03:19:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:19:49.200689537 +0000 UTC m=+86.476260887" watchObservedRunningTime="2026-07-24 03:19:49.20138532 +0000 UTC m=+86.476956870" Jul 24 03:19:50.187909 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:50.187882 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-57ddd6f59f-7nt49" Jul 24 03:19:51.190959 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:51.190918 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"406d6dd0-7f29-4f7c-9061-086ba77be93d","Type":"ContainerStarted","Data":"5d09fe6083926ef2fb931350347f8bac4913d7a8241a866ac8f98ba80aa2d03e"} Jul 24 03:19:52.198593 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:52.198415 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e523c03c-b9eb-44cb-b2fd-967c7b02f14f","Type":"ContainerStarted","Data":"fa728199c80f8784136789e7c1e339365fc49d44302109ff343cd090cbe81673"} Jul 24 03:19:52.198593 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:52.198538 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e523c03c-b9eb-44cb-b2fd-967c7b02f14f","Type":"ContainerStarted","Data":"aebb1d489025a16a6b44aab549a945cc337823083179c49a8d8bb7ffe0f0c348"} Jul 24 03:19:52.203051 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:52.203020 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"406d6dd0-7f29-4f7c-9061-086ba77be93d","Type":"ContainerStarted","Data":"df2f507c389998510f305ad117a71e98a006be25bb768162914c8b61ad697660"} Jul 24 03:19:52.203219 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:52.203100 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"406d6dd0-7f29-4f7c-9061-086ba77be93d","Type":"ContainerStarted","Data":"c69241d13a2903597509e6d3238f502536645ba6a6e8e64c85002fc6199b7bab"} Jul 24 03:19:52.203219 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:52.203144 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"406d6dd0-7f29-4f7c-9061-086ba77be93d","Type":"ContainerStarted","Data":"1d831ab0bf01826a90f6002c0ed208cb7cf190b18c4558e983cdab796eb6a2ee"} Jul 24 03:19:52.203219 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:52.203185 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"406d6dd0-7f29-4f7c-9061-086ba77be93d","Type":"ContainerStarted","Data":"c06b2e44c791e3ea2888faa7f3be3f29b2d36931c2ba27b3570f0d361b1be299"} Jul 24 03:19:52.313343 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:52.313309 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-797fc79d89-jj4f4" Jul 24 03:19:54.214963 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:54.214932 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"406d6dd0-7f29-4f7c-9061-086ba77be93d","Type":"ContainerStarted","Data":"1f4df049d243be48ba2c6cddaf281f58223efc42cbd1c48e57aedf1770121f53"} Jul 24 03:19:54.238828 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:54.238752 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=13.277935584 podStartE2EDuration="29.238685431s" totalImagesPullingTime="15.937866687s" totalInitContainerRuntime="22.88316ms" isStatefulPod=false podCreationTimestamp="2026-07-24 03:19:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:19:54.238557318 +0000 UTC m=+91.514128667" watchObservedRunningTime="2026-07-24 03:19:54.238685431 +0000 UTC m=+91.514256778" Jul 24 03:19:55.225412 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:55.225370 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e523c03c-b9eb-44cb-b2fd-967c7b02f14f","Type":"ContainerStarted","Data":"d3eaf7a077108a771f2702d1e76fe1b107c346b06b28949033cba8c9d91de714"} Jul 24 03:19:55.225999 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:55.225489 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e523c03c-b9eb-44cb-b2fd-967c7b02f14f","Type":"ContainerStarted","Data":"8c7c9f68ad992c4c668fbb55dc0e5c312d707255ce56979c5b6cee5f7b463a1e"} Jul 24 03:19:55.225999 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:55.225533 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e523c03c-b9eb-44cb-b2fd-967c7b02f14f","Type":"ContainerStarted","Data":"afb3edd1f913c47834dc070e1b2433c085cfd2fc5ebe75c4414e7c0d5af2ca3d"} Jul 24 03:19:55.225999 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:55.225579 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e523c03c-b9eb-44cb-b2fd-967c7b02f14f","Type":"ContainerStarted","Data":"256f3218fe513d32629b3996c245aa52b0bb99c1c4f9d4d680647f3a6bcb43c5"} Jul 24 03:19:55.255494 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:55.255428 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=6.600604248 podStartE2EDuration="25.255405751s" totalImagesPullingTime="18.609608804s" totalInitContainerRuntime="45.192699ms" isStatefulPod=false podCreationTimestamp="2026-07-24 03:19:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:19:55.253607187 +0000 UTC m=+92.529178538" watchObservedRunningTime="2026-07-24 03:19:55.255405751 +0000 UTC m=+92.530977103" Jul 24 03:19:55.506079 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:19:55.505994 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:08.622184 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:08.622156 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-d4b79fc88-njng4" Jul 24 03:20:08.626302 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:08.626280 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-d4b79fc88-njng4" Jul 24 03:20:13.180918 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:13.180875 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-797fc79d89-jj4f4" podUID="a32fb5b0-3f50-486a-9f5b-f989b67cd313" containerName="console" containerID="cri-o://bc75cb99e8d5d28b67c182385d48dd3df4d9559069270831209c1ade46d863cd" gracePeriod=15 Jul 24 03:20:13.299466 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:13.299440 2579 generic.go:356] "Generic (PLEG): container finished" podID="a32fb5b0-3f50-486a-9f5b-f989b67cd313" containerID="bc75cb99e8d5d28b67c182385d48dd3df4d9559069270831209c1ade46d863cd" exitCode=0 Jul 24 03:20:13.299573 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:13.299487 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-797fc79d89-jj4f4" event={"ID":"a32fb5b0-3f50-486a-9f5b-f989b67cd313","Type":"ContainerDied","Data":"bc75cb99e8d5d28b67c182385d48dd3df4d9559069270831209c1ade46d863cd"} Jul 24 03:20:13.445602 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:13.445546 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-797fc79d89-jj4f4" Jul 24 03:20:13.588103 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:13.588076 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a32fb5b0-3f50-486a-9f5b-f989b67cd313-service-ca\") pod \"a32fb5b0-3f50-486a-9f5b-f989b67cd313\" (UID: \"a32fb5b0-3f50-486a-9f5b-f989b67cd313\") " Jul 24 03:20:13.588258 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:13.588133 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a32fb5b0-3f50-486a-9f5b-f989b67cd313-oauth-serving-cert\") pod \"a32fb5b0-3f50-486a-9f5b-f989b67cd313\" (UID: \"a32fb5b0-3f50-486a-9f5b-f989b67cd313\") " Jul 24 03:20:13.588258 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:13.588154 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a32fb5b0-3f50-486a-9f5b-f989b67cd313-console-serving-cert\") pod \"a32fb5b0-3f50-486a-9f5b-f989b67cd313\" (UID: \"a32fb5b0-3f50-486a-9f5b-f989b67cd313\") " Jul 24 03:20:13.588258 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:13.588184 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a32fb5b0-3f50-486a-9f5b-f989b67cd313-console-config\") pod \"a32fb5b0-3f50-486a-9f5b-f989b67cd313\" (UID: \"a32fb5b0-3f50-486a-9f5b-f989b67cd313\") " Jul 24 03:20:13.588258 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:13.588249 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4dh9\" (UniqueName: \"kubernetes.io/projected/a32fb5b0-3f50-486a-9f5b-f989b67cd313-kube-api-access-p4dh9\") pod \"a32fb5b0-3f50-486a-9f5b-f989b67cd313\" (UID: \"a32fb5b0-3f50-486a-9f5b-f989b67cd313\") " Jul 24 03:20:13.588471 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:13.588304 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a32fb5b0-3f50-486a-9f5b-f989b67cd313-console-oauth-config\") pod \"a32fb5b0-3f50-486a-9f5b-f989b67cd313\" (UID: \"a32fb5b0-3f50-486a-9f5b-f989b67cd313\") " Jul 24 03:20:13.588636 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:13.588607 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a32fb5b0-3f50-486a-9f5b-f989b67cd313-service-ca" (OuterVolumeSpecName: "service-ca") pod "a32fb5b0-3f50-486a-9f5b-f989b67cd313" (UID: "a32fb5b0-3f50-486a-9f5b-f989b67cd313"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 03:20:13.588741 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:13.588616 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a32fb5b0-3f50-486a-9f5b-f989b67cd313-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a32fb5b0-3f50-486a-9f5b-f989b67cd313" (UID: "a32fb5b0-3f50-486a-9f5b-f989b67cd313"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 03:20:13.588741 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:13.588657 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a32fb5b0-3f50-486a-9f5b-f989b67cd313-console-config" (OuterVolumeSpecName: "console-config") pod "a32fb5b0-3f50-486a-9f5b-f989b67cd313" (UID: "a32fb5b0-3f50-486a-9f5b-f989b67cd313"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 03:20:13.590655 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:13.590609 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a32fb5b0-3f50-486a-9f5b-f989b67cd313-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a32fb5b0-3f50-486a-9f5b-f989b67cd313" (UID: "a32fb5b0-3f50-486a-9f5b-f989b67cd313"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 03:20:13.590655 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:13.590632 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a32fb5b0-3f50-486a-9f5b-f989b67cd313-kube-api-access-p4dh9" (OuterVolumeSpecName: "kube-api-access-p4dh9") pod "a32fb5b0-3f50-486a-9f5b-f989b67cd313" (UID: "a32fb5b0-3f50-486a-9f5b-f989b67cd313"). InnerVolumeSpecName "kube-api-access-p4dh9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 03:20:13.590815 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:13.590697 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a32fb5b0-3f50-486a-9f5b-f989b67cd313-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a32fb5b0-3f50-486a-9f5b-f989b67cd313" (UID: "a32fb5b0-3f50-486a-9f5b-f989b67cd313"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 03:20:13.689320 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:13.689295 2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a32fb5b0-3f50-486a-9f5b-f989b67cd313-service-ca\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:13.689320 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:13.689317 2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a32fb5b0-3f50-486a-9f5b-f989b67cd313-oauth-serving-cert\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:13.689474 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:13.689327 2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a32fb5b0-3f50-486a-9f5b-f989b67cd313-console-serving-cert\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:13.689474 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:13.689336 2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a32fb5b0-3f50-486a-9f5b-f989b67cd313-console-config\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:13.689474 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:13.689344 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4dh9\" (UniqueName: \"kubernetes.io/projected/a32fb5b0-3f50-486a-9f5b-f989b67cd313-kube-api-access-p4dh9\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:13.689474 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:13.689353 2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a32fb5b0-3f50-486a-9f5b-f989b67cd313-console-oauth-config\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:14.304110 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:14.304073 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-797fc79d89-jj4f4" event={"ID":"a32fb5b0-3f50-486a-9f5b-f989b67cd313","Type":"ContainerDied","Data":"c80fbc09e83680f369dd8ef65e017d8c16c9f854ff8ab5a7624704dcd4c41d2c"} Jul 24 03:20:14.304570 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:14.304115 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-797fc79d89-jj4f4" Jul 24 03:20:14.304570 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:14.304171 2579 scope.go:117] "RemoveContainer" containerID="bc75cb99e8d5d28b67c182385d48dd3df4d9559069270831209c1ade46d863cd" Jul 24 03:20:14.323521 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:14.323494 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-797fc79d89-jj4f4"] Jul 24 03:20:14.325787 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:14.325767 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-797fc79d89-jj4f4"] Jul 24 03:20:15.520749 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:15.520720 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a32fb5b0-3f50-486a-9f5b-f989b67cd313" path="/var/lib/kubelet/pods/a32fb5b0-3f50-486a-9f5b-f989b67cd313/volumes" Jul 24 03:20:19.185088 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:19.185059 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-52h8l" Jul 24 03:20:30.506321 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:30.506280 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:30.525070 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:30.525039 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:31.381893 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:31.381861 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:44.246943 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:44.246907 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 03:20:44.247509 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:44.247224 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="406d6dd0-7f29-4f7c-9061-086ba77be93d" containerName="alertmanager" containerID="cri-o://5d09fe6083926ef2fb931350347f8bac4913d7a8241a866ac8f98ba80aa2d03e" gracePeriod=120 Jul 24 03:20:44.247509 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:44.247295 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="406d6dd0-7f29-4f7c-9061-086ba77be93d" containerName="kube-rbac-proxy-metric" containerID="cri-o://df2f507c389998510f305ad117a71e98a006be25bb768162914c8b61ad697660" gracePeriod=120 Jul 24 03:20:44.247509 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:44.247358 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="406d6dd0-7f29-4f7c-9061-086ba77be93d" containerName="kube-rbac-proxy-web" containerID="cri-o://1d831ab0bf01826a90f6002c0ed208cb7cf190b18c4558e983cdab796eb6a2ee" gracePeriod=120 Jul 24 03:20:44.247509 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:44.247397 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="406d6dd0-7f29-4f7c-9061-086ba77be93d" containerName="config-reloader" containerID="cri-o://c06b2e44c791e3ea2888faa7f3be3f29b2d36931c2ba27b3570f0d361b1be299" gracePeriod=120 Jul 24 03:20:44.247795 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:44.247556 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="406d6dd0-7f29-4f7c-9061-086ba77be93d" containerName="kube-rbac-proxy" containerID="cri-o://c69241d13a2903597509e6d3238f502536645ba6a6e8e64c85002fc6199b7bab" gracePeriod=120 Jul 24 03:20:44.247795 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:44.247705 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="406d6dd0-7f29-4f7c-9061-086ba77be93d" containerName="prom-label-proxy" containerID="cri-o://1f4df049d243be48ba2c6cddaf281f58223efc42cbd1c48e57aedf1770121f53" gracePeriod=120 Jul 24 03:20:44.414744 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:44.414719 2579 generic.go:356] "Generic (PLEG): container finished" podID="406d6dd0-7f29-4f7c-9061-086ba77be93d" containerID="1f4df049d243be48ba2c6cddaf281f58223efc42cbd1c48e57aedf1770121f53" exitCode=0 Jul 24 03:20:44.414744 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:44.414741 2579 generic.go:356] "Generic (PLEG): container finished" podID="406d6dd0-7f29-4f7c-9061-086ba77be93d" containerID="c69241d13a2903597509e6d3238f502536645ba6a6e8e64c85002fc6199b7bab" exitCode=0 Jul 24 03:20:44.414744 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:44.414749 2579 generic.go:356] "Generic (PLEG): container finished" podID="406d6dd0-7f29-4f7c-9061-086ba77be93d" containerID="c06b2e44c791e3ea2888faa7f3be3f29b2d36931c2ba27b3570f0d361b1be299" exitCode=0 Jul 24 03:20:44.414921 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:44.414755 2579 generic.go:356] "Generic (PLEG): container finished" podID="406d6dd0-7f29-4f7c-9061-086ba77be93d" containerID="5d09fe6083926ef2fb931350347f8bac4913d7a8241a866ac8f98ba80aa2d03e" exitCode=0 Jul 24 03:20:44.414921 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:44.414804 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"406d6dd0-7f29-4f7c-9061-086ba77be93d","Type":"ContainerDied","Data":"1f4df049d243be48ba2c6cddaf281f58223efc42cbd1c48e57aedf1770121f53"} Jul 24 03:20:44.414921 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:44.414874 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"406d6dd0-7f29-4f7c-9061-086ba77be93d","Type":"ContainerDied","Data":"c69241d13a2903597509e6d3238f502536645ba6a6e8e64c85002fc6199b7bab"} Jul 24 03:20:44.414921 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:44.414901 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"406d6dd0-7f29-4f7c-9061-086ba77be93d","Type":"ContainerDied","Data":"c06b2e44c791e3ea2888faa7f3be3f29b2d36931c2ba27b3570f0d361b1be299"} Jul 24 03:20:44.415042 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:44.414925 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"406d6dd0-7f29-4f7c-9061-086ba77be93d","Type":"ContainerDied","Data":"5d09fe6083926ef2fb931350347f8bac4913d7a8241a866ac8f98ba80aa2d03e"} Jul 24 03:20:45.423786 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.423753 2579 generic.go:356] "Generic (PLEG): container finished" podID="406d6dd0-7f29-4f7c-9061-086ba77be93d" containerID="df2f507c389998510f305ad117a71e98a006be25bb768162914c8b61ad697660" exitCode=0 Jul 24 03:20:45.423786 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.423785 2579 generic.go:356] "Generic (PLEG): container finished" podID="406d6dd0-7f29-4f7c-9061-086ba77be93d" containerID="1d831ab0bf01826a90f6002c0ed208cb7cf190b18c4558e983cdab796eb6a2ee" exitCode=0 Jul 24 03:20:45.424185 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.423822 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"406d6dd0-7f29-4f7c-9061-086ba77be93d","Type":"ContainerDied","Data":"df2f507c389998510f305ad117a71e98a006be25bb768162914c8b61ad697660"} Jul 24 03:20:45.424185 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.423909 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"406d6dd0-7f29-4f7c-9061-086ba77be93d","Type":"ContainerDied","Data":"1d831ab0bf01826a90f6002c0ed208cb7cf190b18c4558e983cdab796eb6a2ee"} Jul 24 03:20:45.498311 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.498286 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:45.545770 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.545736 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/406d6dd0-7f29-4f7c-9061-086ba77be93d-alertmanager-trusted-ca-bundle\") pod \"406d6dd0-7f29-4f7c-9061-086ba77be93d\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " Jul 24 03:20:45.545927 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.545783 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/406d6dd0-7f29-4f7c-9061-086ba77be93d-tls-assets\") pod \"406d6dd0-7f29-4f7c-9061-086ba77be93d\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " Jul 24 03:20:45.545927 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.545804 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"406d6dd0-7f29-4f7c-9061-086ba77be93d\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " Jul 24 03:20:45.546043 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.545951 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-secret-alertmanager-main-tls\") pod \"406d6dd0-7f29-4f7c-9061-086ba77be93d\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " Jul 24 03:20:45.546043 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.545990 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hsmzl\" (UniqueName: \"kubernetes.io/projected/406d6dd0-7f29-4f7c-9061-086ba77be93d-kube-api-access-hsmzl\") pod \"406d6dd0-7f29-4f7c-9061-086ba77be93d\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " Jul 24 03:20:45.546159 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.546077 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-web-config\") pod \"406d6dd0-7f29-4f7c-9061-086ba77be93d\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " Jul 24 03:20:45.546159 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.546130 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-secret-alertmanager-kube-rbac-proxy-web\") pod \"406d6dd0-7f29-4f7c-9061-086ba77be93d\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " Jul 24 03:20:45.546261 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.546146 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/406d6dd0-7f29-4f7c-9061-086ba77be93d-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "406d6dd0-7f29-4f7c-9061-086ba77be93d" (UID: "406d6dd0-7f29-4f7c-9061-086ba77be93d"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 03:20:45.546261 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.546173 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/406d6dd0-7f29-4f7c-9061-086ba77be93d-alertmanager-main-db\") pod \"406d6dd0-7f29-4f7c-9061-086ba77be93d\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " Jul 24 03:20:45.546261 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.546210 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-secret-alertmanager-kube-rbac-proxy\") pod \"406d6dd0-7f29-4f7c-9061-086ba77be93d\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " Jul 24 03:20:45.546261 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.546246 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/406d6dd0-7f29-4f7c-9061-086ba77be93d-metrics-client-ca\") pod \"406d6dd0-7f29-4f7c-9061-086ba77be93d\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " Jul 24 03:20:45.546468 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.546270 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/406d6dd0-7f29-4f7c-9061-086ba77be93d-config-out\") pod \"406d6dd0-7f29-4f7c-9061-086ba77be93d\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " Jul 24 03:20:45.546468 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.546298 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-cluster-tls-config\") pod \"406d6dd0-7f29-4f7c-9061-086ba77be93d\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " Jul 24 03:20:45.546468 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.546329 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-config-volume\") pod \"406d6dd0-7f29-4f7c-9061-086ba77be93d\" (UID: \"406d6dd0-7f29-4f7c-9061-086ba77be93d\") " Jul 24 03:20:45.546609 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.546565 2579 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/406d6dd0-7f29-4f7c-9061-086ba77be93d-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:45.546813 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.546784 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/406d6dd0-7f29-4f7c-9061-086ba77be93d-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "406d6dd0-7f29-4f7c-9061-086ba77be93d" (UID: "406d6dd0-7f29-4f7c-9061-086ba77be93d"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 03:20:45.547233 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.547205 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/406d6dd0-7f29-4f7c-9061-086ba77be93d-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "406d6dd0-7f29-4f7c-9061-086ba77be93d" (UID: "406d6dd0-7f29-4f7c-9061-086ba77be93d"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 03:20:45.548804 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.548770 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/406d6dd0-7f29-4f7c-9061-086ba77be93d-kube-api-access-hsmzl" (OuterVolumeSpecName: "kube-api-access-hsmzl") pod "406d6dd0-7f29-4f7c-9061-086ba77be93d" (UID: "406d6dd0-7f29-4f7c-9061-086ba77be93d"). InnerVolumeSpecName "kube-api-access-hsmzl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 03:20:45.549368 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.549338 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "406d6dd0-7f29-4f7c-9061-086ba77be93d" (UID: "406d6dd0-7f29-4f7c-9061-086ba77be93d"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 03:20:45.550575 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.550549 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/406d6dd0-7f29-4f7c-9061-086ba77be93d-config-out" (OuterVolumeSpecName: "config-out") pod "406d6dd0-7f29-4f7c-9061-086ba77be93d" (UID: "406d6dd0-7f29-4f7c-9061-086ba77be93d"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 03:20:45.551278 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.551252 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/406d6dd0-7f29-4f7c-9061-086ba77be93d-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "406d6dd0-7f29-4f7c-9061-086ba77be93d" (UID: "406d6dd0-7f29-4f7c-9061-086ba77be93d"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 03:20:45.551385 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.551318 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-config-volume" (OuterVolumeSpecName: "config-volume") pod "406d6dd0-7f29-4f7c-9061-086ba77be93d" (UID: "406d6dd0-7f29-4f7c-9061-086ba77be93d"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 03:20:45.551446 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.551419 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "406d6dd0-7f29-4f7c-9061-086ba77be93d" (UID: "406d6dd0-7f29-4f7c-9061-086ba77be93d"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 03:20:45.551569 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.551546 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "406d6dd0-7f29-4f7c-9061-086ba77be93d" (UID: "406d6dd0-7f29-4f7c-9061-086ba77be93d"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 03:20:45.551844 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.551821 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "406d6dd0-7f29-4f7c-9061-086ba77be93d" (UID: "406d6dd0-7f29-4f7c-9061-086ba77be93d"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 03:20:45.554842 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.554815 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "406d6dd0-7f29-4f7c-9061-086ba77be93d" (UID: "406d6dd0-7f29-4f7c-9061-086ba77be93d"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 03:20:45.560948 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.560929 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-web-config" (OuterVolumeSpecName: "web-config") pod "406d6dd0-7f29-4f7c-9061-086ba77be93d" (UID: "406d6dd0-7f29-4f7c-9061-086ba77be93d"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 03:20:45.647033 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.646966 2579 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-web-config\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:45.647033 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.646992 2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:45.647033 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.647007 2579 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/406d6dd0-7f29-4f7c-9061-086ba77be93d-alertmanager-main-db\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:45.647033 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.647019 2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:45.647033 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.647034 2579 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/406d6dd0-7f29-4f7c-9061-086ba77be93d-metrics-client-ca\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:45.647269 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.647047 2579 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/406d6dd0-7f29-4f7c-9061-086ba77be93d-config-out\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:45.647269 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.647058 2579 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-cluster-tls-config\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:45.647269 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.647070 2579 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-config-volume\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:45.647269 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.647081 2579 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/406d6dd0-7f29-4f7c-9061-086ba77be93d-tls-assets\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:45.647269 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.647092 2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:45.647269 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.647105 2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/406d6dd0-7f29-4f7c-9061-086ba77be93d-secret-alertmanager-main-tls\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:45.647269 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:45.647118 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hsmzl\" (UniqueName: \"kubernetes.io/projected/406d6dd0-7f29-4f7c-9061-086ba77be93d-kube-api-access-hsmzl\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:46.430442 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.430403 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"406d6dd0-7f29-4f7c-9061-086ba77be93d","Type":"ContainerDied","Data":"a3a712c147ac63fb4c3e454c2287bb446f3d35d406b9e9f5d5ed5b096858cbdd"} Jul 24 03:20:46.430864 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.430483 2579 scope.go:117] "RemoveContainer" containerID="1f4df049d243be48ba2c6cddaf281f58223efc42cbd1c48e57aedf1770121f53" Jul 24 03:20:46.430864 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.430423 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.437888 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.437869 2579 scope.go:117] "RemoveContainer" containerID="df2f507c389998510f305ad117a71e98a006be25bb768162914c8b61ad697660" Jul 24 03:20:46.444618 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.444600 2579 scope.go:117] "RemoveContainer" containerID="c69241d13a2903597509e6d3238f502536645ba6a6e8e64c85002fc6199b7bab" Jul 24 03:20:46.452459 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.452433 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 03:20:46.452610 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.452599 2579 scope.go:117] "RemoveContainer" containerID="1d831ab0bf01826a90f6002c0ed208cb7cf190b18c4558e983cdab796eb6a2ee" Jul 24 03:20:46.455783 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.455760 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 03:20:46.460127 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.460095 2579 scope.go:117] "RemoveContainer" containerID="c06b2e44c791e3ea2888faa7f3be3f29b2d36931c2ba27b3570f0d361b1be299" Jul 24 03:20:46.466501 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.466487 2579 scope.go:117] "RemoveContainer" containerID="5d09fe6083926ef2fb931350347f8bac4913d7a8241a866ac8f98ba80aa2d03e" Jul 24 03:20:46.473045 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.472940 2579 scope.go:117] "RemoveContainer" containerID="6519e86ed1dcf01b70457703b799e0ef66d3774910f3c7498ac78735a15e105a" Jul 24 03:20:46.481476 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.481455 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 03:20:46.481756 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.481742 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="406d6dd0-7f29-4f7c-9061-086ba77be93d" containerName="alertmanager" Jul 24 03:20:46.481808 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.481759 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="406d6dd0-7f29-4f7c-9061-086ba77be93d" containerName="alertmanager" Jul 24 03:20:46.481808 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.481769 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="406d6dd0-7f29-4f7c-9061-086ba77be93d" containerName="kube-rbac-proxy-web" Jul 24 03:20:46.481808 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.481774 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="406d6dd0-7f29-4f7c-9061-086ba77be93d" containerName="kube-rbac-proxy-web" Jul 24 03:20:46.481808 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.481783 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="406d6dd0-7f29-4f7c-9061-086ba77be93d" containerName="init-config-reloader" Jul 24 03:20:46.481808 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.481789 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="406d6dd0-7f29-4f7c-9061-086ba77be93d" containerName="init-config-reloader" Jul 24 03:20:46.481808 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.481795 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="406d6dd0-7f29-4f7c-9061-086ba77be93d" containerName="kube-rbac-proxy-metric" Jul 24 03:20:46.481808 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.481800 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="406d6dd0-7f29-4f7c-9061-086ba77be93d" containerName="kube-rbac-proxy-metric" Jul 24 03:20:46.481808 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.481810 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="406d6dd0-7f29-4f7c-9061-086ba77be93d" containerName="config-reloader" Jul 24 03:20:46.482035 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.481816 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="406d6dd0-7f29-4f7c-9061-086ba77be93d" containerName="config-reloader" Jul 24 03:20:46.482035 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.481826 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="406d6dd0-7f29-4f7c-9061-086ba77be93d" containerName="prom-label-proxy" Jul 24 03:20:46.482035 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.481831 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="406d6dd0-7f29-4f7c-9061-086ba77be93d" containerName="prom-label-proxy" Jul 24 03:20:46.482035 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.481838 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a32fb5b0-3f50-486a-9f5b-f989b67cd313" containerName="console" Jul 24 03:20:46.482035 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.481842 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a32fb5b0-3f50-486a-9f5b-f989b67cd313" containerName="console" Jul 24 03:20:46.482035 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.481851 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="406d6dd0-7f29-4f7c-9061-086ba77be93d" containerName="kube-rbac-proxy" Jul 24 03:20:46.482035 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.481856 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="406d6dd0-7f29-4f7c-9061-086ba77be93d" containerName="kube-rbac-proxy" Jul 24 03:20:46.482035 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.481894 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="406d6dd0-7f29-4f7c-9061-086ba77be93d" containerName="kube-rbac-proxy-metric" Jul 24 03:20:46.482035 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.481902 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="406d6dd0-7f29-4f7c-9061-086ba77be93d" containerName="alertmanager" Jul 24 03:20:46.482035 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.481908 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a32fb5b0-3f50-486a-9f5b-f989b67cd313" containerName="console" Jul 24 03:20:46.482035 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.481913 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="406d6dd0-7f29-4f7c-9061-086ba77be93d" containerName="config-reloader" Jul 24 03:20:46.482035 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.481918 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="406d6dd0-7f29-4f7c-9061-086ba77be93d" containerName="kube-rbac-proxy" Jul 24 03:20:46.482035 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.481924 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="406d6dd0-7f29-4f7c-9061-086ba77be93d" containerName="kube-rbac-proxy-web" Jul 24 03:20:46.482035 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.481929 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="406d6dd0-7f29-4f7c-9061-086ba77be93d" containerName="prom-label-proxy" Jul 24 03:20:46.486806 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.486789 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.490749 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.490726 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 24 03:20:46.490834 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.490737 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 24 03:20:46.490901 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.490871 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 24 03:20:46.490901 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.490885 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 24 03:20:46.491108 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.491091 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 24 03:20:46.491196 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.491118 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-w2cqv\"" Jul 24 03:20:46.491196 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.491168 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 24 03:20:46.491291 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.491210 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 24 03:20:46.491291 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.491259 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 24 03:20:46.499102 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.495984 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 24 03:20:46.502767 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.502747 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 03:20:46.554008 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.553980 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/be51bc2a-40bf-474d-8a89-cfadc92609a4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.554146 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.554012 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/be51bc2a-40bf-474d-8a89-cfadc92609a4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.554146 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.554029 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/be51bc2a-40bf-474d-8a89-cfadc92609a4-config-volume\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.554146 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.554051 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/be51bc2a-40bf-474d-8a89-cfadc92609a4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.554146 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.554108 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/be51bc2a-40bf-474d-8a89-cfadc92609a4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.554281 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.554163 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/be51bc2a-40bf-474d-8a89-cfadc92609a4-config-out\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.554281 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.554191 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/be51bc2a-40bf-474d-8a89-cfadc92609a4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.554281 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.554212 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be51bc2a-40bf-474d-8a89-cfadc92609a4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.554281 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.554230 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be51bc2a-40bf-474d-8a89-cfadc92609a4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.554281 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.554259 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/be51bc2a-40bf-474d-8a89-cfadc92609a4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.554281 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.554274 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/be51bc2a-40bf-474d-8a89-cfadc92609a4-web-config\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.554452 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.554321 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24rzg\" (UniqueName: \"kubernetes.io/projected/be51bc2a-40bf-474d-8a89-cfadc92609a4-kube-api-access-24rzg\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.554452 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.554374 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/be51bc2a-40bf-474d-8a89-cfadc92609a4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.655452 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.655353 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/be51bc2a-40bf-474d-8a89-cfadc92609a4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.655452 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.655405 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/be51bc2a-40bf-474d-8a89-cfadc92609a4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.655641 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.655530 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/be51bc2a-40bf-474d-8a89-cfadc92609a4-config-out\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.655641 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.655588 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/be51bc2a-40bf-474d-8a89-cfadc92609a4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.655641 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.655623 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be51bc2a-40bf-474d-8a89-cfadc92609a4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.655824 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.655651 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be51bc2a-40bf-474d-8a89-cfadc92609a4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.655824 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.655725 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/be51bc2a-40bf-474d-8a89-cfadc92609a4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.656443 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.656413 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be51bc2a-40bf-474d-8a89-cfadc92609a4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.656577 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.656474 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/be51bc2a-40bf-474d-8a89-cfadc92609a4-web-config\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.656577 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.656511 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24rzg\" (UniqueName: \"kubernetes.io/projected/be51bc2a-40bf-474d-8a89-cfadc92609a4-kube-api-access-24rzg\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.656577 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.656555 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/be51bc2a-40bf-474d-8a89-cfadc92609a4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.656769 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.656582 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be51bc2a-40bf-474d-8a89-cfadc92609a4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.656769 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.656596 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/be51bc2a-40bf-474d-8a89-cfadc92609a4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.656769 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.656649 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/be51bc2a-40bf-474d-8a89-cfadc92609a4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.656769 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.656701 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/be51bc2a-40bf-474d-8a89-cfadc92609a4-config-volume\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.658838 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.658469 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/be51bc2a-40bf-474d-8a89-cfadc92609a4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.658838 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.658471 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/be51bc2a-40bf-474d-8a89-cfadc92609a4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.658838 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.658610 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/be51bc2a-40bf-474d-8a89-cfadc92609a4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.658838 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.658789 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/be51bc2a-40bf-474d-8a89-cfadc92609a4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.659118 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.658838 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/be51bc2a-40bf-474d-8a89-cfadc92609a4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.659118 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.658998 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/be51bc2a-40bf-474d-8a89-cfadc92609a4-config-out\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.659426 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.659394 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/be51bc2a-40bf-474d-8a89-cfadc92609a4-config-volume\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.659539 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.659468 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/be51bc2a-40bf-474d-8a89-cfadc92609a4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.659539 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.659468 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/be51bc2a-40bf-474d-8a89-cfadc92609a4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.660470 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.660451 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/be51bc2a-40bf-474d-8a89-cfadc92609a4-web-config\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.664636 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.664610 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24rzg\" (UniqueName: \"kubernetes.io/projected/be51bc2a-40bf-474d-8a89-cfadc92609a4-kube-api-access-24rzg\") pod \"alertmanager-main-0\" (UID: \"be51bc2a-40bf-474d-8a89-cfadc92609a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.796420 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.796389 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 03:20:46.924922 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:46.924898 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 03:20:46.926717 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:20:46.926691 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe51bc2a_40bf_474d_8a89_cfadc92609a4.slice/crio-a06ebc9fce5b5c6ff9a0198fa36e2ceb8efd6c11e1927459d72b39242fd3ad50 WatchSource:0}: Error finding container a06ebc9fce5b5c6ff9a0198fa36e2ceb8efd6c11e1927459d72b39242fd3ad50: Status 404 returned error can't find the container with id a06ebc9fce5b5c6ff9a0198fa36e2ceb8efd6c11e1927459d72b39242fd3ad50 Jul 24 03:20:47.435891 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:47.435856 2579 generic.go:356] "Generic (PLEG): container finished" podID="be51bc2a-40bf-474d-8a89-cfadc92609a4" containerID="2c0294ef65874bcdc60bccd0d2b7f4b7ab8ade1ba82e8f54a224db27d4ff8feb" exitCode=0 Jul 24 03:20:47.436258 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:47.435903 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"be51bc2a-40bf-474d-8a89-cfadc92609a4","Type":"ContainerDied","Data":"2c0294ef65874bcdc60bccd0d2b7f4b7ab8ade1ba82e8f54a224db27d4ff8feb"} Jul 24 03:20:47.436258 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:47.435956 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"be51bc2a-40bf-474d-8a89-cfadc92609a4","Type":"ContainerStarted","Data":"a06ebc9fce5b5c6ff9a0198fa36e2ceb8efd6c11e1927459d72b39242fd3ad50"} Jul 24 03:20:47.521297 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:47.521270 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="406d6dd0-7f29-4f7c-9061-086ba77be93d" path="/var/lib/kubelet/pods/406d6dd0-7f29-4f7c-9061-086ba77be93d/volumes" Jul 24 03:20:48.444404 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.444373 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"be51bc2a-40bf-474d-8a89-cfadc92609a4","Type":"ContainerStarted","Data":"1999d79b4fc628799434f4cdf324be9eef1087e23f71c8c29a9ddc2ab42e2592"} Jul 24 03:20:48.444760 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.444432 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"be51bc2a-40bf-474d-8a89-cfadc92609a4","Type":"ContainerStarted","Data":"cf7dfae4bdb671f31b0eb23d84dd0e0d1077a441bf42cdb0d7d183ed360e1367"} Jul 24 03:20:48.444760 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.444459 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"be51bc2a-40bf-474d-8a89-cfadc92609a4","Type":"ContainerStarted","Data":"f06bd4ccea130fb4b5ddbba951034b3c0a9287da235e2819bc04c96e47154539"} Jul 24 03:20:48.444760 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.444482 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"be51bc2a-40bf-474d-8a89-cfadc92609a4","Type":"ContainerStarted","Data":"126dab3c1b960253bfcf454cfd35018d1ebde5d8cfe5be59b2d5b0511ac20051"} Jul 24 03:20:48.444760 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.444505 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"be51bc2a-40bf-474d-8a89-cfadc92609a4","Type":"ContainerStarted","Data":"3fbc59d9c51a4a22b02486119368f53406eb4cb690b8c804206bb48a71441ed3"} Jul 24 03:20:48.444760 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.444528 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"be51bc2a-40bf-474d-8a89-cfadc92609a4","Type":"ContainerStarted","Data":"7181100adef4ef1fab36cfb17f70d973bddc941e0b4aab08683c30d7adb291c5"} Jul 24 03:20:48.471990 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.471925 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.458251413 podStartE2EDuration="2.471905157s" totalImagesPullingTime="0s" totalInitContainerRuntime="13.653744ms" isStatefulPod=false podCreationTimestamp="2026-07-24 03:20:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:20:48.469616168 +0000 UTC m=+145.745187518" watchObservedRunningTime="2026-07-24 03:20:48.471905157 +0000 UTC m=+145.747476506" Jul 24 03:20:48.546038 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.546003 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 24 03:20:48.546387 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.546351 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e523c03c-b9eb-44cb-b2fd-967c7b02f14f" containerName="prometheus" containerID="cri-o://aebb1d489025a16a6b44aab549a945cc337823083179c49a8d8bb7ffe0f0c348" gracePeriod=600 Jul 24 03:20:48.546483 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.546374 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e523c03c-b9eb-44cb-b2fd-967c7b02f14f" containerName="config-reloader" containerID="cri-o://fa728199c80f8784136789e7c1e339365fc49d44302109ff343cd090cbe81673" gracePeriod=600 Jul 24 03:20:48.546483 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.546392 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e523c03c-b9eb-44cb-b2fd-967c7b02f14f" containerName="thanos-sidecar" containerID="cri-o://256f3218fe513d32629b3996c245aa52b0bb99c1c4f9d4d680647f3a6bcb43c5" gracePeriod=600 Jul 24 03:20:48.546483 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.546407 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e523c03c-b9eb-44cb-b2fd-967c7b02f14f" containerName="kube-rbac-proxy-web" containerID="cri-o://afb3edd1f913c47834dc070e1b2433c085cfd2fc5ebe75c4414e7c0d5af2ca3d" gracePeriod=600 Jul 24 03:20:48.546641 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.546393 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e523c03c-b9eb-44cb-b2fd-967c7b02f14f" containerName="kube-rbac-proxy" containerID="cri-o://8c7c9f68ad992c4c668fbb55dc0e5c312d707255ce56979c5b6cee5f7b463a1e" gracePeriod=600 Jul 24 03:20:48.546641 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.546439 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e523c03c-b9eb-44cb-b2fd-967c7b02f14f" containerName="kube-rbac-proxy-thanos" containerID="cri-o://d3eaf7a077108a771f2702d1e76fe1b107c346b06b28949033cba8c9d91de714" gracePeriod=600 Jul 24 03:20:48.802998 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.802974 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:48.878122 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.878090 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " Jul 24 03:20:48.878268 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.878131 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-secret-metrics-client-certs\") pod \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " Jul 24 03:20:48.878268 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.878169 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " Jul 24 03:20:48.878268 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.878202 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-prometheus-trusted-ca-bundle\") pod \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " Jul 24 03:20:48.878268 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.878242 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-prometheus-k8s-rulefiles-1\") pod \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " Jul 24 03:20:48.878499 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.878273 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-config-out\") pod \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " Jul 24 03:20:48.878499 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.878303 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-prometheus-k8s-rulefiles-2\") pod \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " Jul 24 03:20:48.878499 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.878332 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-thanos-prometheus-http-client-file\") pod \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " Jul 24 03:20:48.878499 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.878366 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-web-config\") pod \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " Jul 24 03:20:48.878499 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.878389 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-configmap-kubelet-serving-ca-bundle\") pod \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " Jul 24 03:20:48.878499 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.878456 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-prometheus-k8s-db\") pod \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " Jul 24 03:20:48.878499 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.878487 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-config\") pod \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " Jul 24 03:20:48.878861 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.878518 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-secret-kube-rbac-proxy\") pod \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " Jul 24 03:20:48.878861 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.878552 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-prometheus-k8s-rulefiles-0\") pod \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " Jul 24 03:20:48.878861 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.878576 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "e523c03c-b9eb-44cb-b2fd-967c7b02f14f" (UID: "e523c03c-b9eb-44cb-b2fd-967c7b02f14f"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 03:20:48.878861 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.878616 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-secret-prometheus-k8s-tls\") pod \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " Jul 24 03:20:48.878861 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.878642 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-configmap-metrics-client-ca\") pod \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " Jul 24 03:20:48.878861 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.878677 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "e523c03c-b9eb-44cb-b2fd-967c7b02f14f" (UID: "e523c03c-b9eb-44cb-b2fd-967c7b02f14f"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 03:20:48.878861 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.878687 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rdp7x\" (UniqueName: \"kubernetes.io/projected/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-kube-api-access-rdp7x\") pod \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " Jul 24 03:20:48.878861 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.878745 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-secret-grpc-tls\") pod \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " Jul 24 03:20:48.878861 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.878863 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-tls-assets\") pod \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " Jul 24 03:20:48.879276 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.878894 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-configmap-serving-certs-ca-bundle\") pod \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\" (UID: \"e523c03c-b9eb-44cb-b2fd-967c7b02f14f\") " Jul 24 03:20:48.879276 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.878920 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "e523c03c-b9eb-44cb-b2fd-967c7b02f14f" (UID: "e523c03c-b9eb-44cb-b2fd-967c7b02f14f"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 03:20:48.879276 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.879156 2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-prometheus-trusted-ca-bundle\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:48.879276 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.879174 2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:48.879276 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.879190 2579 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:48.880182 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.879858 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "e523c03c-b9eb-44cb-b2fd-967c7b02f14f" (UID: "e523c03c-b9eb-44cb-b2fd-967c7b02f14f"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 03:20:48.881394 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.880457 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "e523c03c-b9eb-44cb-b2fd-967c7b02f14f" (UID: "e523c03c-b9eb-44cb-b2fd-967c7b02f14f"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 03:20:48.881394 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.880742 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "e523c03c-b9eb-44cb-b2fd-967c7b02f14f" (UID: "e523c03c-b9eb-44cb-b2fd-967c7b02f14f"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 03:20:48.881394 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.881010 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "e523c03c-b9eb-44cb-b2fd-967c7b02f14f" (UID: "e523c03c-b9eb-44cb-b2fd-967c7b02f14f"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 03:20:48.881829 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.881758 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "e523c03c-b9eb-44cb-b2fd-967c7b02f14f" (UID: "e523c03c-b9eb-44cb-b2fd-967c7b02f14f"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 03:20:48.882150 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.882029 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "e523c03c-b9eb-44cb-b2fd-967c7b02f14f" (UID: "e523c03c-b9eb-44cb-b2fd-967c7b02f14f"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 03:20:48.882359 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.882328 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "e523c03c-b9eb-44cb-b2fd-967c7b02f14f" (UID: "e523c03c-b9eb-44cb-b2fd-967c7b02f14f"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 03:20:48.882542 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.882494 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-kube-api-access-rdp7x" (OuterVolumeSpecName: "kube-api-access-rdp7x") pod "e523c03c-b9eb-44cb-b2fd-967c7b02f14f" (UID: "e523c03c-b9eb-44cb-b2fd-967c7b02f14f"). InnerVolumeSpecName "kube-api-access-rdp7x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 03:20:48.883224 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.883185 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "e523c03c-b9eb-44cb-b2fd-967c7b02f14f" (UID: "e523c03c-b9eb-44cb-b2fd-967c7b02f14f"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 03:20:48.883354 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.883145 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "e523c03c-b9eb-44cb-b2fd-967c7b02f14f" (UID: "e523c03c-b9eb-44cb-b2fd-967c7b02f14f"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 03:20:48.883589 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.883553 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "e523c03c-b9eb-44cb-b2fd-967c7b02f14f" (UID: "e523c03c-b9eb-44cb-b2fd-967c7b02f14f"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 03:20:48.883848 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.883821 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "e523c03c-b9eb-44cb-b2fd-967c7b02f14f" (UID: "e523c03c-b9eb-44cb-b2fd-967c7b02f14f"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 03:20:48.884082 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.884045 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-config-out" (OuterVolumeSpecName: "config-out") pod "e523c03c-b9eb-44cb-b2fd-967c7b02f14f" (UID: "e523c03c-b9eb-44cb-b2fd-967c7b02f14f"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 03:20:48.884167 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.884086 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "e523c03c-b9eb-44cb-b2fd-967c7b02f14f" (UID: "e523c03c-b9eb-44cb-b2fd-967c7b02f14f"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 03:20:48.884515 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.884494 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-config" (OuterVolumeSpecName: "config") pod "e523c03c-b9eb-44cb-b2fd-967c7b02f14f" (UID: "e523c03c-b9eb-44cb-b2fd-967c7b02f14f"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 03:20:48.884745 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.884722 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "e523c03c-b9eb-44cb-b2fd-967c7b02f14f" (UID: "e523c03c-b9eb-44cb-b2fd-967c7b02f14f"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 03:20:48.893325 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.893304 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-web-config" (OuterVolumeSpecName: "web-config") pod "e523c03c-b9eb-44cb-b2fd-967c7b02f14f" (UID: "e523c03c-b9eb-44cb-b2fd-967c7b02f14f"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 03:20:48.980381 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.980322 2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-prometheus-k8s-db\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:48.980381 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.980345 2579 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-config\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:48.980381 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.980356 2579 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-secret-kube-rbac-proxy\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:48.980381 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.980366 2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:48.980381 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.980374 2579 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-secret-prometheus-k8s-tls\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:48.980381 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.980383 2579 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-configmap-metrics-client-ca\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:48.980612 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.980392 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdp7x\" (UniqueName: \"kubernetes.io/projected/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-kube-api-access-rdp7x\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:48.980612 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.980400 2579 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-secret-grpc-tls\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:48.980612 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.980408 2579 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-tls-assets\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:48.980612 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.980417 2579 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:48.980612 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.980425 2579 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:48.980612 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.980434 2579 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-secret-metrics-client-certs\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:48.980612 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.980443 2579 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:48.980612 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.980451 2579 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-config-out\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:48.980612 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.980460 2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:48.980612 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.980468 2579 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-thanos-prometheus-http-client-file\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:48.980612 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:48.980477 2579 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e523c03c-b9eb-44cb-b2fd-967c7b02f14f-web-config\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:20:49.451255 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.451219 2579 generic.go:356] "Generic (PLEG): container finished" podID="e523c03c-b9eb-44cb-b2fd-967c7b02f14f" containerID="d3eaf7a077108a771f2702d1e76fe1b107c346b06b28949033cba8c9d91de714" exitCode=0 Jul 24 03:20:49.451255 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.451246 2579 generic.go:356] "Generic (PLEG): container finished" podID="e523c03c-b9eb-44cb-b2fd-967c7b02f14f" containerID="8c7c9f68ad992c4c668fbb55dc0e5c312d707255ce56979c5b6cee5f7b463a1e" exitCode=0 Jul 24 03:20:49.451255 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.451254 2579 generic.go:356] "Generic (PLEG): container finished" podID="e523c03c-b9eb-44cb-b2fd-967c7b02f14f" containerID="afb3edd1f913c47834dc070e1b2433c085cfd2fc5ebe75c4414e7c0d5af2ca3d" exitCode=0 Jul 24 03:20:49.451255 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.451262 2579 generic.go:356] "Generic (PLEG): container finished" podID="e523c03c-b9eb-44cb-b2fd-967c7b02f14f" containerID="256f3218fe513d32629b3996c245aa52b0bb99c1c4f9d4d680647f3a6bcb43c5" exitCode=0 Jul 24 03:20:49.451864 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.451267 2579 generic.go:356] "Generic (PLEG): container finished" podID="e523c03c-b9eb-44cb-b2fd-967c7b02f14f" containerID="fa728199c80f8784136789e7c1e339365fc49d44302109ff343cd090cbe81673" exitCode=0 Jul 24 03:20:49.451864 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.451273 2579 generic.go:356] "Generic (PLEG): container finished" podID="e523c03c-b9eb-44cb-b2fd-967c7b02f14f" containerID="aebb1d489025a16a6b44aab549a945cc337823083179c49a8d8bb7ffe0f0c348" exitCode=0 Jul 24 03:20:49.451864 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.451291 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.451864 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.451304 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e523c03c-b9eb-44cb-b2fd-967c7b02f14f","Type":"ContainerDied","Data":"d3eaf7a077108a771f2702d1e76fe1b107c346b06b28949033cba8c9d91de714"} Jul 24 03:20:49.451864 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.451382 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e523c03c-b9eb-44cb-b2fd-967c7b02f14f","Type":"ContainerDied","Data":"8c7c9f68ad992c4c668fbb55dc0e5c312d707255ce56979c5b6cee5f7b463a1e"} Jul 24 03:20:49.451864 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.451412 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e523c03c-b9eb-44cb-b2fd-967c7b02f14f","Type":"ContainerDied","Data":"afb3edd1f913c47834dc070e1b2433c085cfd2fc5ebe75c4414e7c0d5af2ca3d"} Jul 24 03:20:49.451864 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.451446 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e523c03c-b9eb-44cb-b2fd-967c7b02f14f","Type":"ContainerDied","Data":"256f3218fe513d32629b3996c245aa52b0bb99c1c4f9d4d680647f3a6bcb43c5"} Jul 24 03:20:49.451864 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.451461 2579 scope.go:117] "RemoveContainer" containerID="d3eaf7a077108a771f2702d1e76fe1b107c346b06b28949033cba8c9d91de714" Jul 24 03:20:49.451864 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.451470 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e523c03c-b9eb-44cb-b2fd-967c7b02f14f","Type":"ContainerDied","Data":"fa728199c80f8784136789e7c1e339365fc49d44302109ff343cd090cbe81673"} Jul 24 03:20:49.451864 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.451505 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e523c03c-b9eb-44cb-b2fd-967c7b02f14f","Type":"ContainerDied","Data":"aebb1d489025a16a6b44aab549a945cc337823083179c49a8d8bb7ffe0f0c348"} Jul 24 03:20:49.451864 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.451540 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e523c03c-b9eb-44cb-b2fd-967c7b02f14f","Type":"ContainerDied","Data":"5275b15824f50dbcbbe70f7b36b08490d778c035fa7c7cd5a687a619a50dbe77"} Jul 24 03:20:49.460325 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.460304 2579 scope.go:117] "RemoveContainer" containerID="8c7c9f68ad992c4c668fbb55dc0e5c312d707255ce56979c5b6cee5f7b463a1e" Jul 24 03:20:49.467983 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.467966 2579 scope.go:117] "RemoveContainer" containerID="afb3edd1f913c47834dc070e1b2433c085cfd2fc5ebe75c4414e7c0d5af2ca3d" Jul 24 03:20:49.474840 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.474791 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 24 03:20:49.474893 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.474857 2579 scope.go:117] "RemoveContainer" containerID="256f3218fe513d32629b3996c245aa52b0bb99c1c4f9d4d680647f3a6bcb43c5" Jul 24 03:20:49.480132 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.480110 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 24 03:20:49.482378 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.482357 2579 scope.go:117] "RemoveContainer" containerID="fa728199c80f8784136789e7c1e339365fc49d44302109ff343cd090cbe81673" Jul 24 03:20:49.489251 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.489232 2579 scope.go:117] "RemoveContainer" containerID="aebb1d489025a16a6b44aab549a945cc337823083179c49a8d8bb7ffe0f0c348" Jul 24 03:20:49.495854 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.495836 2579 scope.go:117] "RemoveContainer" containerID="e9b9884d2041a304c3bb841c27eec6b14e2a825ef4d402af2d2adb66a264fa9d" Jul 24 03:20:49.502218 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.502156 2579 scope.go:117] "RemoveContainer" containerID="d3eaf7a077108a771f2702d1e76fe1b107c346b06b28949033cba8c9d91de714" Jul 24 03:20:49.502566 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:20:49.502538 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3eaf7a077108a771f2702d1e76fe1b107c346b06b28949033cba8c9d91de714\": container with ID starting with d3eaf7a077108a771f2702d1e76fe1b107c346b06b28949033cba8c9d91de714 not found: ID does not exist" containerID="d3eaf7a077108a771f2702d1e76fe1b107c346b06b28949033cba8c9d91de714" Jul 24 03:20:49.502709 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.502575 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3eaf7a077108a771f2702d1e76fe1b107c346b06b28949033cba8c9d91de714"} err="failed to get container status \"d3eaf7a077108a771f2702d1e76fe1b107c346b06b28949033cba8c9d91de714\": rpc error: code = NotFound desc = could not find container \"d3eaf7a077108a771f2702d1e76fe1b107c346b06b28949033cba8c9d91de714\": container with ID starting with d3eaf7a077108a771f2702d1e76fe1b107c346b06b28949033cba8c9d91de714 not found: ID does not exist" Jul 24 03:20:49.502709 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.502615 2579 scope.go:117] "RemoveContainer" containerID="8c7c9f68ad992c4c668fbb55dc0e5c312d707255ce56979c5b6cee5f7b463a1e" Jul 24 03:20:49.502912 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:20:49.502896 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c7c9f68ad992c4c668fbb55dc0e5c312d707255ce56979c5b6cee5f7b463a1e\": container with ID starting with 8c7c9f68ad992c4c668fbb55dc0e5c312d707255ce56979c5b6cee5f7b463a1e not found: ID does not exist" containerID="8c7c9f68ad992c4c668fbb55dc0e5c312d707255ce56979c5b6cee5f7b463a1e" Jul 24 03:20:49.502961 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.502925 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c7c9f68ad992c4c668fbb55dc0e5c312d707255ce56979c5b6cee5f7b463a1e"} err="failed to get container status \"8c7c9f68ad992c4c668fbb55dc0e5c312d707255ce56979c5b6cee5f7b463a1e\": rpc error: code = NotFound desc = could not find container \"8c7c9f68ad992c4c668fbb55dc0e5c312d707255ce56979c5b6cee5f7b463a1e\": container with ID starting with 8c7c9f68ad992c4c668fbb55dc0e5c312d707255ce56979c5b6cee5f7b463a1e not found: ID does not exist" Jul 24 03:20:49.502961 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.502946 2579 scope.go:117] "RemoveContainer" containerID="afb3edd1f913c47834dc070e1b2433c085cfd2fc5ebe75c4414e7c0d5af2ca3d" Jul 24 03:20:49.503231 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:20:49.503190 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afb3edd1f913c47834dc070e1b2433c085cfd2fc5ebe75c4414e7c0d5af2ca3d\": container with ID starting with afb3edd1f913c47834dc070e1b2433c085cfd2fc5ebe75c4414e7c0d5af2ca3d not found: ID does not exist" containerID="afb3edd1f913c47834dc070e1b2433c085cfd2fc5ebe75c4414e7c0d5af2ca3d" Jul 24 03:20:49.503294 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.503240 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afb3edd1f913c47834dc070e1b2433c085cfd2fc5ebe75c4414e7c0d5af2ca3d"} err="failed to get container status \"afb3edd1f913c47834dc070e1b2433c085cfd2fc5ebe75c4414e7c0d5af2ca3d\": rpc error: code = NotFound desc = could not find container \"afb3edd1f913c47834dc070e1b2433c085cfd2fc5ebe75c4414e7c0d5af2ca3d\": container with ID starting with afb3edd1f913c47834dc070e1b2433c085cfd2fc5ebe75c4414e7c0d5af2ca3d not found: ID does not exist" Jul 24 03:20:49.503294 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.503264 2579 scope.go:117] "RemoveContainer" containerID="256f3218fe513d32629b3996c245aa52b0bb99c1c4f9d4d680647f3a6bcb43c5" Jul 24 03:20:49.503474 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:20:49.503456 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"256f3218fe513d32629b3996c245aa52b0bb99c1c4f9d4d680647f3a6bcb43c5\": container with ID starting with 256f3218fe513d32629b3996c245aa52b0bb99c1c4f9d4d680647f3a6bcb43c5 not found: ID does not exist" containerID="256f3218fe513d32629b3996c245aa52b0bb99c1c4f9d4d680647f3a6bcb43c5" Jul 24 03:20:49.503523 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.503482 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"256f3218fe513d32629b3996c245aa52b0bb99c1c4f9d4d680647f3a6bcb43c5"} err="failed to get container status \"256f3218fe513d32629b3996c245aa52b0bb99c1c4f9d4d680647f3a6bcb43c5\": rpc error: code = NotFound desc = could not find container \"256f3218fe513d32629b3996c245aa52b0bb99c1c4f9d4d680647f3a6bcb43c5\": container with ID starting with 256f3218fe513d32629b3996c245aa52b0bb99c1c4f9d4d680647f3a6bcb43c5 not found: ID does not exist" Jul 24 03:20:49.503523 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.503500 2579 scope.go:117] "RemoveContainer" containerID="fa728199c80f8784136789e7c1e339365fc49d44302109ff343cd090cbe81673" Jul 24 03:20:49.503745 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:20:49.503727 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa728199c80f8784136789e7c1e339365fc49d44302109ff343cd090cbe81673\": container with ID starting with fa728199c80f8784136789e7c1e339365fc49d44302109ff343cd090cbe81673 not found: ID does not exist" containerID="fa728199c80f8784136789e7c1e339365fc49d44302109ff343cd090cbe81673" Jul 24 03:20:49.503815 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.503752 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa728199c80f8784136789e7c1e339365fc49d44302109ff343cd090cbe81673"} err="failed to get container status \"fa728199c80f8784136789e7c1e339365fc49d44302109ff343cd090cbe81673\": rpc error: code = NotFound desc = could not find container \"fa728199c80f8784136789e7c1e339365fc49d44302109ff343cd090cbe81673\": container with ID starting with fa728199c80f8784136789e7c1e339365fc49d44302109ff343cd090cbe81673 not found: ID does not exist" Jul 24 03:20:49.503815 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.503772 2579 scope.go:117] "RemoveContainer" containerID="aebb1d489025a16a6b44aab549a945cc337823083179c49a8d8bb7ffe0f0c348" Jul 24 03:20:49.503899 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.503816 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 24 03:20:49.504026 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:20:49.504008 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aebb1d489025a16a6b44aab549a945cc337823083179c49a8d8bb7ffe0f0c348\": container with ID starting with aebb1d489025a16a6b44aab549a945cc337823083179c49a8d8bb7ffe0f0c348 not found: ID does not exist" containerID="aebb1d489025a16a6b44aab549a945cc337823083179c49a8d8bb7ffe0f0c348" Jul 24 03:20:49.504070 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.504031 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aebb1d489025a16a6b44aab549a945cc337823083179c49a8d8bb7ffe0f0c348"} err="failed to get container status \"aebb1d489025a16a6b44aab549a945cc337823083179c49a8d8bb7ffe0f0c348\": rpc error: code = NotFound desc = could not find container \"aebb1d489025a16a6b44aab549a945cc337823083179c49a8d8bb7ffe0f0c348\": container with ID starting with aebb1d489025a16a6b44aab549a945cc337823083179c49a8d8bb7ffe0f0c348 not found: ID does not exist" Jul 24 03:20:49.504070 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.504044 2579 scope.go:117] "RemoveContainer" containerID="e9b9884d2041a304c3bb841c27eec6b14e2a825ef4d402af2d2adb66a264fa9d" Jul 24 03:20:49.504070 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.504065 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e523c03c-b9eb-44cb-b2fd-967c7b02f14f" containerName="kube-rbac-proxy-web" Jul 24 03:20:49.504185 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.504077 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e523c03c-b9eb-44cb-b2fd-967c7b02f14f" containerName="kube-rbac-proxy-web" Jul 24 03:20:49.504185 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.504095 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e523c03c-b9eb-44cb-b2fd-967c7b02f14f" containerName="prometheus" Jul 24 03:20:49.504185 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.504101 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e523c03c-b9eb-44cb-b2fd-967c7b02f14f" containerName="prometheus" Jul 24 03:20:49.504185 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.504113 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e523c03c-b9eb-44cb-b2fd-967c7b02f14f" containerName="init-config-reloader" Jul 24 03:20:49.504185 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.504121 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e523c03c-b9eb-44cb-b2fd-967c7b02f14f" containerName="init-config-reloader" Jul 24 03:20:49.504185 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.504127 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e523c03c-b9eb-44cb-b2fd-967c7b02f14f" containerName="config-reloader" Jul 24 03:20:49.504185 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.504132 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e523c03c-b9eb-44cb-b2fd-967c7b02f14f" containerName="config-reloader" Jul 24 03:20:49.504185 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.504140 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e523c03c-b9eb-44cb-b2fd-967c7b02f14f" containerName="thanos-sidecar" Jul 24 03:20:49.504185 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.504147 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e523c03c-b9eb-44cb-b2fd-967c7b02f14f" containerName="thanos-sidecar" Jul 24 03:20:49.504185 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.504156 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e523c03c-b9eb-44cb-b2fd-967c7b02f14f" containerName="kube-rbac-proxy" Jul 24 03:20:49.504185 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.504162 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e523c03c-b9eb-44cb-b2fd-967c7b02f14f" containerName="kube-rbac-proxy" Jul 24 03:20:49.504185 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.504171 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e523c03c-b9eb-44cb-b2fd-967c7b02f14f" containerName="kube-rbac-proxy-thanos" Jul 24 03:20:49.504185 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.504178 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e523c03c-b9eb-44cb-b2fd-967c7b02f14f" containerName="kube-rbac-proxy-thanos" Jul 24 03:20:49.504625 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.504232 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e523c03c-b9eb-44cb-b2fd-967c7b02f14f" containerName="thanos-sidecar" Jul 24 03:20:49.504625 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.504243 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e523c03c-b9eb-44cb-b2fd-967c7b02f14f" containerName="kube-rbac-proxy-web" Jul 24 03:20:49.504625 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.504253 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e523c03c-b9eb-44cb-b2fd-967c7b02f14f" containerName="config-reloader" Jul 24 03:20:49.504625 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.504262 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e523c03c-b9eb-44cb-b2fd-967c7b02f14f" containerName="prometheus" Jul 24 03:20:49.504625 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.504273 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e523c03c-b9eb-44cb-b2fd-967c7b02f14f" containerName="kube-rbac-proxy" Jul 24 03:20:49.504625 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.504285 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e523c03c-b9eb-44cb-b2fd-967c7b02f14f" containerName="kube-rbac-proxy-thanos" Jul 24 03:20:49.504625 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:20:49.504268 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9b9884d2041a304c3bb841c27eec6b14e2a825ef4d402af2d2adb66a264fa9d\": container with ID starting with e9b9884d2041a304c3bb841c27eec6b14e2a825ef4d402af2d2adb66a264fa9d not found: ID does not exist" containerID="e9b9884d2041a304c3bb841c27eec6b14e2a825ef4d402af2d2adb66a264fa9d" Jul 24 03:20:49.504625 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.504359 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9b9884d2041a304c3bb841c27eec6b14e2a825ef4d402af2d2adb66a264fa9d"} err="failed to get container status \"e9b9884d2041a304c3bb841c27eec6b14e2a825ef4d402af2d2adb66a264fa9d\": rpc error: code = NotFound desc = could not find container \"e9b9884d2041a304c3bb841c27eec6b14e2a825ef4d402af2d2adb66a264fa9d\": container with ID starting with e9b9884d2041a304c3bb841c27eec6b14e2a825ef4d402af2d2adb66a264fa9d not found: ID does not exist" Jul 24 03:20:49.504625 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.504373 2579 scope.go:117] "RemoveContainer" containerID="d3eaf7a077108a771f2702d1e76fe1b107c346b06b28949033cba8c9d91de714" Jul 24 03:20:49.505074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.504626 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3eaf7a077108a771f2702d1e76fe1b107c346b06b28949033cba8c9d91de714"} err="failed to get container status \"d3eaf7a077108a771f2702d1e76fe1b107c346b06b28949033cba8c9d91de714\": rpc error: code = NotFound desc = could not find container \"d3eaf7a077108a771f2702d1e76fe1b107c346b06b28949033cba8c9d91de714\": container with ID starting with d3eaf7a077108a771f2702d1e76fe1b107c346b06b28949033cba8c9d91de714 not found: ID does not exist" Jul 24 03:20:49.505074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.504641 2579 scope.go:117] "RemoveContainer" containerID="8c7c9f68ad992c4c668fbb55dc0e5c312d707255ce56979c5b6cee5f7b463a1e" Jul 24 03:20:49.505074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.504883 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c7c9f68ad992c4c668fbb55dc0e5c312d707255ce56979c5b6cee5f7b463a1e"} err="failed to get container status \"8c7c9f68ad992c4c668fbb55dc0e5c312d707255ce56979c5b6cee5f7b463a1e\": rpc error: code = NotFound desc = could not find container \"8c7c9f68ad992c4c668fbb55dc0e5c312d707255ce56979c5b6cee5f7b463a1e\": container with ID starting with 8c7c9f68ad992c4c668fbb55dc0e5c312d707255ce56979c5b6cee5f7b463a1e not found: ID does not exist" Jul 24 03:20:49.505074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.504900 2579 scope.go:117] "RemoveContainer" containerID="afb3edd1f913c47834dc070e1b2433c085cfd2fc5ebe75c4414e7c0d5af2ca3d" Jul 24 03:20:49.505198 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.505098 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afb3edd1f913c47834dc070e1b2433c085cfd2fc5ebe75c4414e7c0d5af2ca3d"} err="failed to get container status \"afb3edd1f913c47834dc070e1b2433c085cfd2fc5ebe75c4414e7c0d5af2ca3d\": rpc error: code = NotFound desc = could not find container \"afb3edd1f913c47834dc070e1b2433c085cfd2fc5ebe75c4414e7c0d5af2ca3d\": container with ID starting with afb3edd1f913c47834dc070e1b2433c085cfd2fc5ebe75c4414e7c0d5af2ca3d not found: ID does not exist" Jul 24 03:20:49.505198 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.505119 2579 scope.go:117] "RemoveContainer" containerID="256f3218fe513d32629b3996c245aa52b0bb99c1c4f9d4d680647f3a6bcb43c5" Jul 24 03:20:49.505374 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.505355 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"256f3218fe513d32629b3996c245aa52b0bb99c1c4f9d4d680647f3a6bcb43c5"} err="failed to get container status \"256f3218fe513d32629b3996c245aa52b0bb99c1c4f9d4d680647f3a6bcb43c5\": rpc error: code = NotFound desc = could not find container \"256f3218fe513d32629b3996c245aa52b0bb99c1c4f9d4d680647f3a6bcb43c5\": container with ID starting with 256f3218fe513d32629b3996c245aa52b0bb99c1c4f9d4d680647f3a6bcb43c5 not found: ID does not exist" Jul 24 03:20:49.505416 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.505375 2579 scope.go:117] "RemoveContainer" containerID="fa728199c80f8784136789e7c1e339365fc49d44302109ff343cd090cbe81673" Jul 24 03:20:49.505584 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.505566 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa728199c80f8784136789e7c1e339365fc49d44302109ff343cd090cbe81673"} err="failed to get container status \"fa728199c80f8784136789e7c1e339365fc49d44302109ff343cd090cbe81673\": rpc error: code = NotFound desc = could not find container \"fa728199c80f8784136789e7c1e339365fc49d44302109ff343cd090cbe81673\": container with ID starting with fa728199c80f8784136789e7c1e339365fc49d44302109ff343cd090cbe81673 not found: ID does not exist" Jul 24 03:20:49.505635 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.505584 2579 scope.go:117] "RemoveContainer" containerID="aebb1d489025a16a6b44aab549a945cc337823083179c49a8d8bb7ffe0f0c348" Jul 24 03:20:49.505868 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.505851 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aebb1d489025a16a6b44aab549a945cc337823083179c49a8d8bb7ffe0f0c348"} err="failed to get container status \"aebb1d489025a16a6b44aab549a945cc337823083179c49a8d8bb7ffe0f0c348\": rpc error: code = NotFound desc = could not find container \"aebb1d489025a16a6b44aab549a945cc337823083179c49a8d8bb7ffe0f0c348\": container with ID starting with aebb1d489025a16a6b44aab549a945cc337823083179c49a8d8bb7ffe0f0c348 not found: ID does not exist" Jul 24 03:20:49.505931 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.505868 2579 scope.go:117] "RemoveContainer" containerID="e9b9884d2041a304c3bb841c27eec6b14e2a825ef4d402af2d2adb66a264fa9d" Jul 24 03:20:49.506071 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.506055 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9b9884d2041a304c3bb841c27eec6b14e2a825ef4d402af2d2adb66a264fa9d"} err="failed to get container status \"e9b9884d2041a304c3bb841c27eec6b14e2a825ef4d402af2d2adb66a264fa9d\": rpc error: code = NotFound desc = could not find container \"e9b9884d2041a304c3bb841c27eec6b14e2a825ef4d402af2d2adb66a264fa9d\": container with ID starting with e9b9884d2041a304c3bb841c27eec6b14e2a825ef4d402af2d2adb66a264fa9d not found: ID does not exist" Jul 24 03:20:49.506127 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.506072 2579 scope.go:117] "RemoveContainer" containerID="d3eaf7a077108a771f2702d1e76fe1b107c346b06b28949033cba8c9d91de714" Jul 24 03:20:49.506321 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.506291 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3eaf7a077108a771f2702d1e76fe1b107c346b06b28949033cba8c9d91de714"} err="failed to get container status \"d3eaf7a077108a771f2702d1e76fe1b107c346b06b28949033cba8c9d91de714\": rpc error: code = NotFound desc = could not find container \"d3eaf7a077108a771f2702d1e76fe1b107c346b06b28949033cba8c9d91de714\": container with ID starting with d3eaf7a077108a771f2702d1e76fe1b107c346b06b28949033cba8c9d91de714 not found: ID does not exist" Jul 24 03:20:49.506321 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.506321 2579 scope.go:117] "RemoveContainer" containerID="8c7c9f68ad992c4c668fbb55dc0e5c312d707255ce56979c5b6cee5f7b463a1e" Jul 24 03:20:49.506587 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.506566 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c7c9f68ad992c4c668fbb55dc0e5c312d707255ce56979c5b6cee5f7b463a1e"} err="failed to get container status \"8c7c9f68ad992c4c668fbb55dc0e5c312d707255ce56979c5b6cee5f7b463a1e\": rpc error: code = NotFound desc = could not find container \"8c7c9f68ad992c4c668fbb55dc0e5c312d707255ce56979c5b6cee5f7b463a1e\": container with ID starting with 8c7c9f68ad992c4c668fbb55dc0e5c312d707255ce56979c5b6cee5f7b463a1e not found: ID does not exist" Jul 24 03:20:49.506587 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.506586 2579 scope.go:117] "RemoveContainer" containerID="afb3edd1f913c47834dc070e1b2433c085cfd2fc5ebe75c4414e7c0d5af2ca3d" Jul 24 03:20:49.506851 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.506828 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afb3edd1f913c47834dc070e1b2433c085cfd2fc5ebe75c4414e7c0d5af2ca3d"} err="failed to get container status \"afb3edd1f913c47834dc070e1b2433c085cfd2fc5ebe75c4414e7c0d5af2ca3d\": rpc error: code = NotFound desc = could not find container \"afb3edd1f913c47834dc070e1b2433c085cfd2fc5ebe75c4414e7c0d5af2ca3d\": container with ID starting with afb3edd1f913c47834dc070e1b2433c085cfd2fc5ebe75c4414e7c0d5af2ca3d not found: ID does not exist" Jul 24 03:20:49.506851 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.506845 2579 scope.go:117] "RemoveContainer" containerID="256f3218fe513d32629b3996c245aa52b0bb99c1c4f9d4d680647f3a6bcb43c5" Jul 24 03:20:49.507099 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.507079 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"256f3218fe513d32629b3996c245aa52b0bb99c1c4f9d4d680647f3a6bcb43c5"} err="failed to get container status \"256f3218fe513d32629b3996c245aa52b0bb99c1c4f9d4d680647f3a6bcb43c5\": rpc error: code = NotFound desc = could not find container \"256f3218fe513d32629b3996c245aa52b0bb99c1c4f9d4d680647f3a6bcb43c5\": container with ID starting with 256f3218fe513d32629b3996c245aa52b0bb99c1c4f9d4d680647f3a6bcb43c5 not found: ID does not exist" Jul 24 03:20:49.507099 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.507100 2579 scope.go:117] "RemoveContainer" containerID="fa728199c80f8784136789e7c1e339365fc49d44302109ff343cd090cbe81673" Jul 24 03:20:49.507393 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.507375 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa728199c80f8784136789e7c1e339365fc49d44302109ff343cd090cbe81673"} err="failed to get container status \"fa728199c80f8784136789e7c1e339365fc49d44302109ff343cd090cbe81673\": rpc error: code = NotFound desc = could not find container \"fa728199c80f8784136789e7c1e339365fc49d44302109ff343cd090cbe81673\": container with ID starting with fa728199c80f8784136789e7c1e339365fc49d44302109ff343cd090cbe81673 not found: ID does not exist" Jul 24 03:20:49.507465 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.507394 2579 scope.go:117] "RemoveContainer" containerID="aebb1d489025a16a6b44aab549a945cc337823083179c49a8d8bb7ffe0f0c348" Jul 24 03:20:49.507635 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.507616 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aebb1d489025a16a6b44aab549a945cc337823083179c49a8d8bb7ffe0f0c348"} err="failed to get container status \"aebb1d489025a16a6b44aab549a945cc337823083179c49a8d8bb7ffe0f0c348\": rpc error: code = NotFound desc = could not find container \"aebb1d489025a16a6b44aab549a945cc337823083179c49a8d8bb7ffe0f0c348\": container with ID starting with aebb1d489025a16a6b44aab549a945cc337823083179c49a8d8bb7ffe0f0c348 not found: ID does not exist" Jul 24 03:20:49.507703 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.507636 2579 scope.go:117] "RemoveContainer" containerID="e9b9884d2041a304c3bb841c27eec6b14e2a825ef4d402af2d2adb66a264fa9d" Jul 24 03:20:49.507929 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.507909 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9b9884d2041a304c3bb841c27eec6b14e2a825ef4d402af2d2adb66a264fa9d"} err="failed to get container status \"e9b9884d2041a304c3bb841c27eec6b14e2a825ef4d402af2d2adb66a264fa9d\": rpc error: code = NotFound desc = could not find container \"e9b9884d2041a304c3bb841c27eec6b14e2a825ef4d402af2d2adb66a264fa9d\": container with ID starting with e9b9884d2041a304c3bb841c27eec6b14e2a825ef4d402af2d2adb66a264fa9d not found: ID does not exist" Jul 24 03:20:49.507929 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.507927 2579 scope.go:117] "RemoveContainer" containerID="d3eaf7a077108a771f2702d1e76fe1b107c346b06b28949033cba8c9d91de714" Jul 24 03:20:49.508158 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.508135 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3eaf7a077108a771f2702d1e76fe1b107c346b06b28949033cba8c9d91de714"} err="failed to get container status \"d3eaf7a077108a771f2702d1e76fe1b107c346b06b28949033cba8c9d91de714\": rpc error: code = NotFound desc = could not find container \"d3eaf7a077108a771f2702d1e76fe1b107c346b06b28949033cba8c9d91de714\": container with ID starting with d3eaf7a077108a771f2702d1e76fe1b107c346b06b28949033cba8c9d91de714 not found: ID does not exist" Jul 24 03:20:49.508158 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.508158 2579 scope.go:117] "RemoveContainer" containerID="8c7c9f68ad992c4c668fbb55dc0e5c312d707255ce56979c5b6cee5f7b463a1e" Jul 24 03:20:49.508307 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.508291 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.508386 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.508366 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c7c9f68ad992c4c668fbb55dc0e5c312d707255ce56979c5b6cee5f7b463a1e"} err="failed to get container status \"8c7c9f68ad992c4c668fbb55dc0e5c312d707255ce56979c5b6cee5f7b463a1e\": rpc error: code = NotFound desc = could not find container \"8c7c9f68ad992c4c668fbb55dc0e5c312d707255ce56979c5b6cee5f7b463a1e\": container with ID starting with 8c7c9f68ad992c4c668fbb55dc0e5c312d707255ce56979c5b6cee5f7b463a1e not found: ID does not exist" Jul 24 03:20:49.508436 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.508388 2579 scope.go:117] "RemoveContainer" containerID="afb3edd1f913c47834dc070e1b2433c085cfd2fc5ebe75c4414e7c0d5af2ca3d" Jul 24 03:20:49.508723 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.508608 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afb3edd1f913c47834dc070e1b2433c085cfd2fc5ebe75c4414e7c0d5af2ca3d"} err="failed to get container status \"afb3edd1f913c47834dc070e1b2433c085cfd2fc5ebe75c4414e7c0d5af2ca3d\": rpc error: code = NotFound desc = could not find container \"afb3edd1f913c47834dc070e1b2433c085cfd2fc5ebe75c4414e7c0d5af2ca3d\": container with ID starting with afb3edd1f913c47834dc070e1b2433c085cfd2fc5ebe75c4414e7c0d5af2ca3d not found: ID does not exist" Jul 24 03:20:49.508723 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.508638 2579 scope.go:117] "RemoveContainer" containerID="256f3218fe513d32629b3996c245aa52b0bb99c1c4f9d4d680647f3a6bcb43c5" Jul 24 03:20:49.508995 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.508973 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"256f3218fe513d32629b3996c245aa52b0bb99c1c4f9d4d680647f3a6bcb43c5"} err="failed to get container status \"256f3218fe513d32629b3996c245aa52b0bb99c1c4f9d4d680647f3a6bcb43c5\": rpc error: code = NotFound desc = could not find container \"256f3218fe513d32629b3996c245aa52b0bb99c1c4f9d4d680647f3a6bcb43c5\": container with ID starting with 256f3218fe513d32629b3996c245aa52b0bb99c1c4f9d4d680647f3a6bcb43c5 not found: ID does not exist" Jul 24 03:20:49.508995 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.508994 2579 scope.go:117] "RemoveContainer" containerID="fa728199c80f8784136789e7c1e339365fc49d44302109ff343cd090cbe81673" Jul 24 03:20:49.509277 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.509261 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa728199c80f8784136789e7c1e339365fc49d44302109ff343cd090cbe81673"} err="failed to get container status \"fa728199c80f8784136789e7c1e339365fc49d44302109ff343cd090cbe81673\": rpc error: code = NotFound desc = could not find container \"fa728199c80f8784136789e7c1e339365fc49d44302109ff343cd090cbe81673\": container with ID starting with fa728199c80f8784136789e7c1e339365fc49d44302109ff343cd090cbe81673 not found: ID does not exist" Jul 24 03:20:49.509320 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.509279 2579 scope.go:117] "RemoveContainer" containerID="aebb1d489025a16a6b44aab549a945cc337823083179c49a8d8bb7ffe0f0c348" Jul 24 03:20:49.509699 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.509652 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aebb1d489025a16a6b44aab549a945cc337823083179c49a8d8bb7ffe0f0c348"} err="failed to get container status \"aebb1d489025a16a6b44aab549a945cc337823083179c49a8d8bb7ffe0f0c348\": rpc error: code = NotFound desc = could not find container \"aebb1d489025a16a6b44aab549a945cc337823083179c49a8d8bb7ffe0f0c348\": container with ID starting with aebb1d489025a16a6b44aab549a945cc337823083179c49a8d8bb7ffe0f0c348 not found: ID does not exist" Jul 24 03:20:49.509797 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.509698 2579 scope.go:117] "RemoveContainer" containerID="e9b9884d2041a304c3bb841c27eec6b14e2a825ef4d402af2d2adb66a264fa9d" Jul 24 03:20:49.509953 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.509935 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9b9884d2041a304c3bb841c27eec6b14e2a825ef4d402af2d2adb66a264fa9d"} err="failed to get container status \"e9b9884d2041a304c3bb841c27eec6b14e2a825ef4d402af2d2adb66a264fa9d\": rpc error: code = NotFound desc = could not find container \"e9b9884d2041a304c3bb841c27eec6b14e2a825ef4d402af2d2adb66a264fa9d\": container with ID starting with e9b9884d2041a304c3bb841c27eec6b14e2a825ef4d402af2d2adb66a264fa9d not found: ID does not exist" Jul 24 03:20:49.510003 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.509954 2579 scope.go:117] "RemoveContainer" containerID="d3eaf7a077108a771f2702d1e76fe1b107c346b06b28949033cba8c9d91de714" Jul 24 03:20:49.510216 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.510197 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3eaf7a077108a771f2702d1e76fe1b107c346b06b28949033cba8c9d91de714"} err="failed to get container status \"d3eaf7a077108a771f2702d1e76fe1b107c346b06b28949033cba8c9d91de714\": rpc error: code = NotFound desc = could not find container \"d3eaf7a077108a771f2702d1e76fe1b107c346b06b28949033cba8c9d91de714\": container with ID starting with d3eaf7a077108a771f2702d1e76fe1b107c346b06b28949033cba8c9d91de714 not found: ID does not exist" Jul 24 03:20:49.510216 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.510216 2579 scope.go:117] "RemoveContainer" containerID="8c7c9f68ad992c4c668fbb55dc0e5c312d707255ce56979c5b6cee5f7b463a1e" Jul 24 03:20:49.510448 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.510434 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 24 03:20:49.510521 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.510468 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c7c9f68ad992c4c668fbb55dc0e5c312d707255ce56979c5b6cee5f7b463a1e"} err="failed to get container status \"8c7c9f68ad992c4c668fbb55dc0e5c312d707255ce56979c5b6cee5f7b463a1e\": rpc error: code = NotFound desc = could not find container \"8c7c9f68ad992c4c668fbb55dc0e5c312d707255ce56979c5b6cee5f7b463a1e\": container with ID starting with 8c7c9f68ad992c4c668fbb55dc0e5c312d707255ce56979c5b6cee5f7b463a1e not found: ID does not exist" Jul 24 03:20:49.510521 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.510491 2579 scope.go:117] "RemoveContainer" containerID="afb3edd1f913c47834dc070e1b2433c085cfd2fc5ebe75c4414e7c0d5af2ca3d" Jul 24 03:20:49.510763 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.510737 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afb3edd1f913c47834dc070e1b2433c085cfd2fc5ebe75c4414e7c0d5af2ca3d"} err="failed to get container status \"afb3edd1f913c47834dc070e1b2433c085cfd2fc5ebe75c4414e7c0d5af2ca3d\": rpc error: code = NotFound desc = could not find container \"afb3edd1f913c47834dc070e1b2433c085cfd2fc5ebe75c4414e7c0d5af2ca3d\": container with ID starting with afb3edd1f913c47834dc070e1b2433c085cfd2fc5ebe75c4414e7c0d5af2ca3d not found: ID does not exist" Jul 24 03:20:49.510763 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.510756 2579 scope.go:117] "RemoveContainer" containerID="256f3218fe513d32629b3996c245aa52b0bb99c1c4f9d4d680647f3a6bcb43c5" Jul 24 03:20:49.510902 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.510777 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 24 03:20:49.510902 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.510875 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 24 03:20:49.511045 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.511020 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"256f3218fe513d32629b3996c245aa52b0bb99c1c4f9d4d680647f3a6bcb43c5"} err="failed to get container status \"256f3218fe513d32629b3996c245aa52b0bb99c1c4f9d4d680647f3a6bcb43c5\": rpc error: code = NotFound desc = could not find container \"256f3218fe513d32629b3996c245aa52b0bb99c1c4f9d4d680647f3a6bcb43c5\": container with ID starting with 256f3218fe513d32629b3996c245aa52b0bb99c1c4f9d4d680647f3a6bcb43c5 not found: ID does not exist" Jul 24 03:20:49.511105 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.511047 2579 scope.go:117] "RemoveContainer" containerID="fa728199c80f8784136789e7c1e339365fc49d44302109ff343cd090cbe81673" Jul 24 03:20:49.511173 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.511158 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 24 03:20:49.511223 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.511192 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 24 03:20:49.511273 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.511253 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 24 03:20:49.511324 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.511280 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa728199c80f8784136789e7c1e339365fc49d44302109ff343cd090cbe81673"} err="failed to get container status \"fa728199c80f8784136789e7c1e339365fc49d44302109ff343cd090cbe81673\": rpc error: code = NotFound desc = could not find container \"fa728199c80f8784136789e7c1e339365fc49d44302109ff343cd090cbe81673\": container with ID starting with fa728199c80f8784136789e7c1e339365fc49d44302109ff343cd090cbe81673 not found: ID does not exist" Jul 24 03:20:49.511324 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.511305 2579 scope.go:117] "RemoveContainer" containerID="aebb1d489025a16a6b44aab549a945cc337823083179c49a8d8bb7ffe0f0c348" Jul 24 03:20:49.511429 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.511336 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 24 03:20:49.511577 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.511550 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 24 03:20:49.511716 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.511605 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aebb1d489025a16a6b44aab549a945cc337823083179c49a8d8bb7ffe0f0c348"} err="failed to get container status \"aebb1d489025a16a6b44aab549a945cc337823083179c49a8d8bb7ffe0f0c348\": rpc error: code = NotFound desc = could not find container \"aebb1d489025a16a6b44aab549a945cc337823083179c49a8d8bb7ffe0f0c348\": container with ID starting with aebb1d489025a16a6b44aab549a945cc337823083179c49a8d8bb7ffe0f0c348 not found: ID does not exist" Jul 24 03:20:49.511716 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.511627 2579 scope.go:117] "RemoveContainer" containerID="e9b9884d2041a304c3bb841c27eec6b14e2a825ef4d402af2d2adb66a264fa9d" Jul 24 03:20:49.511716 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.511674 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-ztqd4\"" Jul 24 03:20:49.512074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.511769 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 24 03:20:49.512074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.511870 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 24 03:20:49.512074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.511870 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 24 03:20:49.512074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.512017 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 24 03:20:49.512074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.512039 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9b9884d2041a304c3bb841c27eec6b14e2a825ef4d402af2d2adb66a264fa9d"} err="failed to get container status \"e9b9884d2041a304c3bb841c27eec6b14e2a825ef4d402af2d2adb66a264fa9d\": rpc error: code = NotFound desc = could not find container \"e9b9884d2041a304c3bb841c27eec6b14e2a825ef4d402af2d2adb66a264fa9d\": container with ID starting with e9b9884d2041a304c3bb841c27eec6b14e2a825ef4d402af2d2adb66a264fa9d not found: ID does not exist" Jul 24 03:20:49.512074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.512060 2579 scope.go:117] "RemoveContainer" containerID="d3eaf7a077108a771f2702d1e76fe1b107c346b06b28949033cba8c9d91de714" Jul 24 03:20:49.512608 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.512448 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3eaf7a077108a771f2702d1e76fe1b107c346b06b28949033cba8c9d91de714"} err="failed to get container status \"d3eaf7a077108a771f2702d1e76fe1b107c346b06b28949033cba8c9d91de714\": rpc error: code = NotFound desc = could not find container \"d3eaf7a077108a771f2702d1e76fe1b107c346b06b28949033cba8c9d91de714\": container with ID starting with d3eaf7a077108a771f2702d1e76fe1b107c346b06b28949033cba8c9d91de714 not found: ID does not exist" Jul 24 03:20:49.512608 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.512464 2579 scope.go:117] "RemoveContainer" containerID="8c7c9f68ad992c4c668fbb55dc0e5c312d707255ce56979c5b6cee5f7b463a1e" Jul 24 03:20:49.512803 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.512698 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6rqb5c3grvcau\"" Jul 24 03:20:49.512803 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.512711 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c7c9f68ad992c4c668fbb55dc0e5c312d707255ce56979c5b6cee5f7b463a1e"} err="failed to get container status \"8c7c9f68ad992c4c668fbb55dc0e5c312d707255ce56979c5b6cee5f7b463a1e\": rpc error: code = NotFound desc = could not find container \"8c7c9f68ad992c4c668fbb55dc0e5c312d707255ce56979c5b6cee5f7b463a1e\": container with ID starting with 8c7c9f68ad992c4c668fbb55dc0e5c312d707255ce56979c5b6cee5f7b463a1e not found: ID does not exist" Jul 24 03:20:49.512803 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.512734 2579 scope.go:117] "RemoveContainer" containerID="afb3edd1f913c47834dc070e1b2433c085cfd2fc5ebe75c4414e7c0d5af2ca3d" Jul 24 03:20:49.513062 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.513025 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afb3edd1f913c47834dc070e1b2433c085cfd2fc5ebe75c4414e7c0d5af2ca3d"} err="failed to get container status \"afb3edd1f913c47834dc070e1b2433c085cfd2fc5ebe75c4414e7c0d5af2ca3d\": rpc error: code = NotFound desc = could not find container \"afb3edd1f913c47834dc070e1b2433c085cfd2fc5ebe75c4414e7c0d5af2ca3d\": container with ID starting with afb3edd1f913c47834dc070e1b2433c085cfd2fc5ebe75c4414e7c0d5af2ca3d not found: ID does not exist" Jul 24 03:20:49.513062 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.513043 2579 scope.go:117] "RemoveContainer" containerID="256f3218fe513d32629b3996c245aa52b0bb99c1c4f9d4d680647f3a6bcb43c5" Jul 24 03:20:49.513325 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.513300 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"256f3218fe513d32629b3996c245aa52b0bb99c1c4f9d4d680647f3a6bcb43c5"} err="failed to get container status \"256f3218fe513d32629b3996c245aa52b0bb99c1c4f9d4d680647f3a6bcb43c5\": rpc error: code = NotFound desc = could not find container \"256f3218fe513d32629b3996c245aa52b0bb99c1c4f9d4d680647f3a6bcb43c5\": container with ID starting with 256f3218fe513d32629b3996c245aa52b0bb99c1c4f9d4d680647f3a6bcb43c5 not found: ID does not exist" Jul 24 03:20:49.513416 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.513326 2579 scope.go:117] "RemoveContainer" containerID="fa728199c80f8784136789e7c1e339365fc49d44302109ff343cd090cbe81673" Jul 24 03:20:49.513654 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.513631 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa728199c80f8784136789e7c1e339365fc49d44302109ff343cd090cbe81673"} err="failed to get container status \"fa728199c80f8784136789e7c1e339365fc49d44302109ff343cd090cbe81673\": rpc error: code = NotFound desc = could not find container \"fa728199c80f8784136789e7c1e339365fc49d44302109ff343cd090cbe81673\": container with ID starting with fa728199c80f8784136789e7c1e339365fc49d44302109ff343cd090cbe81673 not found: ID does not exist" Jul 24 03:20:49.513744 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.513657 2579 scope.go:117] "RemoveContainer" containerID="aebb1d489025a16a6b44aab549a945cc337823083179c49a8d8bb7ffe0f0c348" Jul 24 03:20:49.514019 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.513999 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aebb1d489025a16a6b44aab549a945cc337823083179c49a8d8bb7ffe0f0c348"} err="failed to get container status \"aebb1d489025a16a6b44aab549a945cc337823083179c49a8d8bb7ffe0f0c348\": rpc error: code = NotFound desc = could not find container \"aebb1d489025a16a6b44aab549a945cc337823083179c49a8d8bb7ffe0f0c348\": container with ID starting with aebb1d489025a16a6b44aab549a945cc337823083179c49a8d8bb7ffe0f0c348 not found: ID does not exist" Jul 24 03:20:49.514109 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.514022 2579 scope.go:117] "RemoveContainer" containerID="e9b9884d2041a304c3bb841c27eec6b14e2a825ef4d402af2d2adb66a264fa9d" Jul 24 03:20:49.514220 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.514201 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 24 03:20:49.514301 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.514282 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9b9884d2041a304c3bb841c27eec6b14e2a825ef4d402af2d2adb66a264fa9d"} err="failed to get container status \"e9b9884d2041a304c3bb841c27eec6b14e2a825ef4d402af2d2adb66a264fa9d\": rpc error: code = NotFound desc = could not find container \"e9b9884d2041a304c3bb841c27eec6b14e2a825ef4d402af2d2adb66a264fa9d\": container with ID starting with e9b9884d2041a304c3bb841c27eec6b14e2a825ef4d402af2d2adb66a264fa9d not found: ID does not exist" Jul 24 03:20:49.517467 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.517447 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 24 03:20:49.520533 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.520513 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e523c03c-b9eb-44cb-b2fd-967c7b02f14f" path="/var/lib/kubelet/pods/e523c03c-b9eb-44cb-b2fd-967c7b02f14f/volumes" Jul 24 03:20:49.526148 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.526125 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 24 03:20:49.583680 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.583643 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/348f762b-b7d0-4be5-af57-41b58d6140f8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.583784 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.583694 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/348f762b-b7d0-4be5-af57-41b58d6140f8-config\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.583784 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.583721 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/348f762b-b7d0-4be5-af57-41b58d6140f8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.583784 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.583738 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/348f762b-b7d0-4be5-af57-41b58d6140f8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.583784 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.583758 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zkvs\" (UniqueName: \"kubernetes.io/projected/348f762b-b7d0-4be5-af57-41b58d6140f8-kube-api-access-2zkvs\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.583784 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.583778 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/348f762b-b7d0-4be5-af57-41b58d6140f8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.584010 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.583876 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/348f762b-b7d0-4be5-af57-41b58d6140f8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.584010 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.583907 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/348f762b-b7d0-4be5-af57-41b58d6140f8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.584010 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.583934 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/348f762b-b7d0-4be5-af57-41b58d6140f8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.584010 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.583958 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/348f762b-b7d0-4be5-af57-41b58d6140f8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.584010 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.583995 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/348f762b-b7d0-4be5-af57-41b58d6140f8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.584254 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.584044 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/348f762b-b7d0-4be5-af57-41b58d6140f8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.584254 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.584098 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/348f762b-b7d0-4be5-af57-41b58d6140f8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.584254 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.584128 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/348f762b-b7d0-4be5-af57-41b58d6140f8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.584254 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.584154 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/348f762b-b7d0-4be5-af57-41b58d6140f8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.584254 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.584182 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/348f762b-b7d0-4be5-af57-41b58d6140f8-web-config\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.584254 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.584210 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/348f762b-b7d0-4be5-af57-41b58d6140f8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.584254 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.584233 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/348f762b-b7d0-4be5-af57-41b58d6140f8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.584501 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.584261 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/348f762b-b7d0-4be5-af57-41b58d6140f8-config-out\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.584501 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.584286 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/348f762b-b7d0-4be5-af57-41b58d6140f8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.685513 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.685444 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/348f762b-b7d0-4be5-af57-41b58d6140f8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.685513 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.685477 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/348f762b-b7d0-4be5-af57-41b58d6140f8-config\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.685513 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.685507 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/348f762b-b7d0-4be5-af57-41b58d6140f8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.685809 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.685620 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/348f762b-b7d0-4be5-af57-41b58d6140f8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.685809 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.685647 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zkvs\" (UniqueName: \"kubernetes.io/projected/348f762b-b7d0-4be5-af57-41b58d6140f8-kube-api-access-2zkvs\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.685809 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.685687 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/348f762b-b7d0-4be5-af57-41b58d6140f8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.685809 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.685728 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/348f762b-b7d0-4be5-af57-41b58d6140f8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.685809 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.685765 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/348f762b-b7d0-4be5-af57-41b58d6140f8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.686126 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.686045 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/348f762b-b7d0-4be5-af57-41b58d6140f8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.686126 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.686099 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/348f762b-b7d0-4be5-af57-41b58d6140f8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.686126 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.686110 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/348f762b-b7d0-4be5-af57-41b58d6140f8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.686126 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.686125 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/348f762b-b7d0-4be5-af57-41b58d6140f8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.686324 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.686167 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/348f762b-b7d0-4be5-af57-41b58d6140f8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.686324 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.686208 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/348f762b-b7d0-4be5-af57-41b58d6140f8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.686324 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.686244 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/348f762b-b7d0-4be5-af57-41b58d6140f8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.686324 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.686273 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/348f762b-b7d0-4be5-af57-41b58d6140f8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.686324 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.686303 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/348f762b-b7d0-4be5-af57-41b58d6140f8-web-config\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.686564 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.686331 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/348f762b-b7d0-4be5-af57-41b58d6140f8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.686564 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.686340 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/348f762b-b7d0-4be5-af57-41b58d6140f8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.686683 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.686355 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/348f762b-b7d0-4be5-af57-41b58d6140f8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.687374 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.686850 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/348f762b-b7d0-4be5-af57-41b58d6140f8-config-out\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.687374 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.686913 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/348f762b-b7d0-4be5-af57-41b58d6140f8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.687374 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.687320 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/348f762b-b7d0-4be5-af57-41b58d6140f8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.688219 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.687639 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/348f762b-b7d0-4be5-af57-41b58d6140f8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.688790 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.688764 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/348f762b-b7d0-4be5-af57-41b58d6140f8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.688896 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.688787 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/348f762b-b7d0-4be5-af57-41b58d6140f8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.689631 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.689608 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/348f762b-b7d0-4be5-af57-41b58d6140f8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.690460 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.689901 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/348f762b-b7d0-4be5-af57-41b58d6140f8-config-out\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.692642 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.690633 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/348f762b-b7d0-4be5-af57-41b58d6140f8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.692642 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.691328 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/348f762b-b7d0-4be5-af57-41b58d6140f8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.692642 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.691696 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/348f762b-b7d0-4be5-af57-41b58d6140f8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.692642 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.691884 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/348f762b-b7d0-4be5-af57-41b58d6140f8-web-config\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.692642 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.692445 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/348f762b-b7d0-4be5-af57-41b58d6140f8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.695944 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.693163 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/348f762b-b7d0-4be5-af57-41b58d6140f8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.695944 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.693395 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/348f762b-b7d0-4be5-af57-41b58d6140f8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.695944 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.693427 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/348f762b-b7d0-4be5-af57-41b58d6140f8-config\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.695944 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.693772 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/348f762b-b7d0-4be5-af57-41b58d6140f8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.695944 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.693894 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/348f762b-b7d0-4be5-af57-41b58d6140f8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.695944 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.695010 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/348f762b-b7d0-4be5-af57-41b58d6140f8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.697580 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.697562 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zkvs\" (UniqueName: \"kubernetes.io/projected/348f762b-b7d0-4be5-af57-41b58d6140f8-kube-api-access-2zkvs\") pod \"prometheus-k8s-0\" (UID: \"348f762b-b7d0-4be5-af57-41b58d6140f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.818937 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.818912 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:20:49.949397 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:49.949375 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 24 03:20:49.951740 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:20:49.951720 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod348f762b_b7d0_4be5_af57_41b58d6140f8.slice/crio-607175eca85e2cecac09ae4961925ebd269667391689f79ec5f51280126c3a29 WatchSource:0}: Error finding container 607175eca85e2cecac09ae4961925ebd269667391689f79ec5f51280126c3a29: Status 404 returned error can't find the container with id 607175eca85e2cecac09ae4961925ebd269667391689f79ec5f51280126c3a29 Jul 24 03:20:50.457047 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:50.457015 2579 generic.go:356] "Generic (PLEG): container finished" podID="348f762b-b7d0-4be5-af57-41b58d6140f8" containerID="666f104818f632aaf9653663d308493c44b238e8d0f6b601c8b34d56032868e2" exitCode=0 Jul 24 03:20:50.457433 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:50.457101 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"348f762b-b7d0-4be5-af57-41b58d6140f8","Type":"ContainerDied","Data":"666f104818f632aaf9653663d308493c44b238e8d0f6b601c8b34d56032868e2"} Jul 24 03:20:50.457433 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:50.457187 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"348f762b-b7d0-4be5-af57-41b58d6140f8","Type":"ContainerStarted","Data":"607175eca85e2cecac09ae4961925ebd269667391689f79ec5f51280126c3a29"} Jul 24 03:20:51.465236 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:51.465197 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"348f762b-b7d0-4be5-af57-41b58d6140f8","Type":"ContainerStarted","Data":"ad911b63c048922bae1e6fdeb88b9647f318f35fe97aaa0cd96e92c8e795d2f4"} Jul 24 03:20:51.465601 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:51.465276 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"348f762b-b7d0-4be5-af57-41b58d6140f8","Type":"ContainerStarted","Data":"1503a97f16403549e73f89864834de656dd9672763908ef892227c457126be3e"} Jul 24 03:20:51.465601 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:51.465315 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"348f762b-b7d0-4be5-af57-41b58d6140f8","Type":"ContainerStarted","Data":"39c8eba5eaf92e56a7441c0955924750c36d741db4f6f278ab0c81fce8eb7433"} Jul 24 03:20:51.465601 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:51.465358 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"348f762b-b7d0-4be5-af57-41b58d6140f8","Type":"ContainerStarted","Data":"5349fc7191b9e9d36349bc784bed42cd186747c83fbd5d31a40d69d3241558f7"} Jul 24 03:20:51.465601 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:51.465393 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"348f762b-b7d0-4be5-af57-41b58d6140f8","Type":"ContainerStarted","Data":"d492ed16bf94cbf08296dd758c3f46dbf45e2e2c117c49cf69c80ce119b97edf"} Jul 24 03:20:51.465601 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:51.465428 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"348f762b-b7d0-4be5-af57-41b58d6140f8","Type":"ContainerStarted","Data":"d98da9349213fc39b5a94c9663eed6bd135e773dba6683f2a1c561e246c09bd8"} Jul 24 03:20:51.493034 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:51.492982 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.477125272 podStartE2EDuration="2.492961708s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.836436ms" isStatefulPod=false podCreationTimestamp="2026-07-24 03:20:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:20:51.491164417 +0000 UTC m=+148.766735767" watchObservedRunningTime="2026-07-24 03:20:51.492961708 +0000 UTC m=+148.768533058" Jul 24 03:20:54.819327 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:20:54.819292 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:21:03.837602 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:21:03.837569 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57ddd6f59f-7nt49"] Jul 24 03:21:28.855611 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:21:28.855554 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-57ddd6f59f-7nt49" podUID="6f61b295-42d3-45a4-9437-a6ec0e961459" containerName="console" containerID="cri-o://1a39958685134a59689d266501274902dfc4933543064b1c7499ac5b5b6b410a" gracePeriod=15 Jul 24 03:21:29.096587 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:21:29.096562 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57ddd6f59f-7nt49" Jul 24 03:21:29.112197 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:21:29.112155 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f61b295-42d3-45a4-9437-a6ec0e961459-oauth-serving-cert\") pod \"6f61b295-42d3-45a4-9437-a6ec0e961459\" (UID: \"6f61b295-42d3-45a4-9437-a6ec0e961459\") " Jul 24 03:21:29.112259 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:21:29.112207 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f61b295-42d3-45a4-9437-a6ec0e961459-console-config\") pod \"6f61b295-42d3-45a4-9437-a6ec0e961459\" (UID: \"6f61b295-42d3-45a4-9437-a6ec0e961459\") " Jul 24 03:21:29.112259 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:21:29.112237 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f61b295-42d3-45a4-9437-a6ec0e961459-console-oauth-config\") pod \"6f61b295-42d3-45a4-9437-a6ec0e961459\" (UID: \"6f61b295-42d3-45a4-9437-a6ec0e961459\") " Jul 24 03:21:29.112346 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:21:29.112269 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f61b295-42d3-45a4-9437-a6ec0e961459-trusted-ca-bundle\") pod \"6f61b295-42d3-45a4-9437-a6ec0e961459\" (UID: \"6f61b295-42d3-45a4-9437-a6ec0e961459\") " Jul 24 03:21:29.112346 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:21:29.112291 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f61b295-42d3-45a4-9437-a6ec0e961459-console-serving-cert\") pod \"6f61b295-42d3-45a4-9437-a6ec0e961459\" (UID: \"6f61b295-42d3-45a4-9437-a6ec0e961459\") " Jul 24 03:21:29.112346 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:21:29.112311 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f61b295-42d3-45a4-9437-a6ec0e961459-service-ca\") pod \"6f61b295-42d3-45a4-9437-a6ec0e961459\" (UID: \"6f61b295-42d3-45a4-9437-a6ec0e961459\") " Jul 24 03:21:29.112346 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:21:29.112336 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dszfd\" (UniqueName: \"kubernetes.io/projected/6f61b295-42d3-45a4-9437-a6ec0e961459-kube-api-access-dszfd\") pod \"6f61b295-42d3-45a4-9437-a6ec0e961459\" (UID: \"6f61b295-42d3-45a4-9437-a6ec0e961459\") " Jul 24 03:21:29.112570 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:21:29.112536 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f61b295-42d3-45a4-9437-a6ec0e961459-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6f61b295-42d3-45a4-9437-a6ec0e961459" (UID: "6f61b295-42d3-45a4-9437-a6ec0e961459"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 03:21:29.112815 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:21:29.112789 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f61b295-42d3-45a4-9437-a6ec0e961459-console-config" (OuterVolumeSpecName: "console-config") pod "6f61b295-42d3-45a4-9437-a6ec0e961459" (UID: "6f61b295-42d3-45a4-9437-a6ec0e961459"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 03:21:29.112913 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:21:29.112793 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f61b295-42d3-45a4-9437-a6ec0e961459-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6f61b295-42d3-45a4-9437-a6ec0e961459" (UID: "6f61b295-42d3-45a4-9437-a6ec0e961459"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 03:21:29.112913 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:21:29.112857 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f61b295-42d3-45a4-9437-a6ec0e961459-service-ca" (OuterVolumeSpecName: "service-ca") pod "6f61b295-42d3-45a4-9437-a6ec0e961459" (UID: "6f61b295-42d3-45a4-9437-a6ec0e961459"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 03:21:29.114364 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:21:29.114339 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f61b295-42d3-45a4-9437-a6ec0e961459-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6f61b295-42d3-45a4-9437-a6ec0e961459" (UID: "6f61b295-42d3-45a4-9437-a6ec0e961459"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 03:21:29.114483 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:21:29.114458 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f61b295-42d3-45a4-9437-a6ec0e961459-kube-api-access-dszfd" (OuterVolumeSpecName: "kube-api-access-dszfd") pod "6f61b295-42d3-45a4-9437-a6ec0e961459" (UID: "6f61b295-42d3-45a4-9437-a6ec0e961459"). InnerVolumeSpecName "kube-api-access-dszfd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 03:21:29.114532 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:21:29.114500 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f61b295-42d3-45a4-9437-a6ec0e961459-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6f61b295-42d3-45a4-9437-a6ec0e961459" (UID: "6f61b295-42d3-45a4-9437-a6ec0e961459"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 03:21:29.212980 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:21:29.212949 2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f61b295-42d3-45a4-9437-a6ec0e961459-trusted-ca-bundle\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:21:29.212980 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:21:29.212975 2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f61b295-42d3-45a4-9437-a6ec0e961459-console-serving-cert\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:21:29.212980 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:21:29.212985 2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f61b295-42d3-45a4-9437-a6ec0e961459-service-ca\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:21:29.213215 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:21:29.212996 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dszfd\" (UniqueName: \"kubernetes.io/projected/6f61b295-42d3-45a4-9437-a6ec0e961459-kube-api-access-dszfd\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:21:29.213215 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:21:29.213005 2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f61b295-42d3-45a4-9437-a6ec0e961459-oauth-serving-cert\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:21:29.213215 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:21:29.213013 2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f61b295-42d3-45a4-9437-a6ec0e961459-console-config\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:21:29.213215 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:21:29.213021 2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f61b295-42d3-45a4-9437-a6ec0e961459-console-oauth-config\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:21:29.606204 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:21:29.606174 2579 generic.go:356] "Generic (PLEG): container finished" podID="6f61b295-42d3-45a4-9437-a6ec0e961459" containerID="1a39958685134a59689d266501274902dfc4933543064b1c7499ac5b5b6b410a" exitCode=0 Jul 24 03:21:29.606380 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:21:29.606220 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57ddd6f59f-7nt49" Jul 24 03:21:29.606380 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:21:29.606268 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57ddd6f59f-7nt49" event={"ID":"6f61b295-42d3-45a4-9437-a6ec0e961459","Type":"ContainerDied","Data":"1a39958685134a59689d266501274902dfc4933543064b1c7499ac5b5b6b410a"} Jul 24 03:21:29.606380 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:21:29.606322 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57ddd6f59f-7nt49" event={"ID":"6f61b295-42d3-45a4-9437-a6ec0e961459","Type":"ContainerDied","Data":"a2ed95b181328d6ee1d2f14d3b6f3d382eb65ea6ed261fcdd46c14bbeef41334"} Jul 24 03:21:29.606380 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:21:29.606347 2579 scope.go:117] "RemoveContainer" containerID="1a39958685134a59689d266501274902dfc4933543064b1c7499ac5b5b6b410a" Jul 24 03:21:29.614528 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:21:29.614512 2579 scope.go:117] "RemoveContainer" containerID="1a39958685134a59689d266501274902dfc4933543064b1c7499ac5b5b6b410a" Jul 24 03:21:29.614787 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:21:29.614767 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a39958685134a59689d266501274902dfc4933543064b1c7499ac5b5b6b410a\": container with ID starting with 1a39958685134a59689d266501274902dfc4933543064b1c7499ac5b5b6b410a not found: ID does not exist" containerID="1a39958685134a59689d266501274902dfc4933543064b1c7499ac5b5b6b410a" Jul 24 03:21:29.614860 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:21:29.614798 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a39958685134a59689d266501274902dfc4933543064b1c7499ac5b5b6b410a"} err="failed to get container status \"1a39958685134a59689d266501274902dfc4933543064b1c7499ac5b5b6b410a\": rpc error: code = NotFound desc = could not find container \"1a39958685134a59689d266501274902dfc4933543064b1c7499ac5b5b6b410a\": container with ID starting with 1a39958685134a59689d266501274902dfc4933543064b1c7499ac5b5b6b410a not found: ID does not exist" Jul 24 03:21:29.622119 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:21:29.622097 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57ddd6f59f-7nt49"] Jul 24 03:21:29.625564 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:21:29.625544 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-57ddd6f59f-7nt49"] Jul 24 03:21:31.520015 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:21:31.519983 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f61b295-42d3-45a4-9437-a6ec0e961459" path="/var/lib/kubelet/pods/6f61b295-42d3-45a4-9437-a6ec0e961459/volumes" Jul 24 03:21:49.819418 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:21:49.819388 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:21:49.834503 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:21:49.834477 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:21:50.699456 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:21:50.699431 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 03:23:23.300129 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:23.300100 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ffrlv_6b2e05ef-119c-4450-9827-f806c437f29f/ovn-acl-logging/0.log" Jul 24 03:23:23.305599 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:23.305572 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ffrlv_6b2e05ef-119c-4450-9827-f806c437f29f/ovn-acl-logging/0.log" Jul 24 03:23:23.307650 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:23.307631 2579 kubelet.go:1648] "Image garbage collection succeeded" Jul 24 03:23:36.956855 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:36.956822 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-znp5n"] Jul 24 03:23:36.959297 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:36.957116 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f61b295-42d3-45a4-9437-a6ec0e961459" containerName="console" Jul 24 03:23:36.959297 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:36.957128 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f61b295-42d3-45a4-9437-a6ec0e961459" containerName="console" Jul 24 03:23:36.959297 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:36.957172 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f61b295-42d3-45a4-9437-a6ec0e961459" containerName="console" Jul 24 03:23:36.960107 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:36.960092 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-znp5n" Jul 24 03:23:36.962254 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:36.962234 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 24 03:23:36.962354 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:36.962323 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-xl4s9\"" Jul 24 03:23:36.962857 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:36.962842 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 24 03:23:36.969467 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:36.969447 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-znp5n"] Jul 24 03:23:37.036294 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:37.036268 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/22921ac9-5226-4222-861e-1412f2515c1f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-znp5n\" (UID: \"22921ac9-5226-4222-861e-1412f2515c1f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-znp5n" Jul 24 03:23:37.036426 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:37.036307 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fb4hw\" (UniqueName: \"kubernetes.io/projected/22921ac9-5226-4222-861e-1412f2515c1f-kube-api-access-fb4hw\") pod \"cert-manager-cainjector-559cdd8c87-znp5n\" (UID: \"22921ac9-5226-4222-861e-1412f2515c1f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-znp5n" Jul 24 03:23:37.137334 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:37.137308 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/22921ac9-5226-4222-861e-1412f2515c1f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-znp5n\" (UID: \"22921ac9-5226-4222-861e-1412f2515c1f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-znp5n" Jul 24 03:23:37.137453 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:37.137349 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fb4hw\" (UniqueName: \"kubernetes.io/projected/22921ac9-5226-4222-861e-1412f2515c1f-kube-api-access-fb4hw\") pod \"cert-manager-cainjector-559cdd8c87-znp5n\" (UID: \"22921ac9-5226-4222-861e-1412f2515c1f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-znp5n" Jul 24 03:23:37.145044 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:37.145021 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/22921ac9-5226-4222-861e-1412f2515c1f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-znp5n\" (UID: \"22921ac9-5226-4222-861e-1412f2515c1f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-znp5n" Jul 24 03:23:37.145178 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:37.145159 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fb4hw\" (UniqueName: \"kubernetes.io/projected/22921ac9-5226-4222-861e-1412f2515c1f-kube-api-access-fb4hw\") pod \"cert-manager-cainjector-559cdd8c87-znp5n\" (UID: \"22921ac9-5226-4222-861e-1412f2515c1f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-znp5n" Jul 24 03:23:37.282002 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:37.281943 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-znp5n" Jul 24 03:23:37.416214 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:37.416188 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-znp5n"] Jul 24 03:23:37.418303 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:23:37.418274 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod22921ac9_5226_4222_861e_1412f2515c1f.slice/crio-0712eee369b4266f7a457fe08bf5fb1e206508ae3d57a2fbf6dca65e9c319804 WatchSource:0}: Error finding container 0712eee369b4266f7a457fe08bf5fb1e206508ae3d57a2fbf6dca65e9c319804: Status 404 returned error can't find the container with id 0712eee369b4266f7a457fe08bf5fb1e206508ae3d57a2fbf6dca65e9c319804 Jul 24 03:23:37.420266 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:37.420246 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 03:23:38.082544 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:38.082510 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-znp5n" event={"ID":"22921ac9-5226-4222-861e-1412f2515c1f","Type":"ContainerStarted","Data":"0712eee369b4266f7a457fe08bf5fb1e206508ae3d57a2fbf6dca65e9c319804"} Jul 24 03:23:41.096314 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:41.096276 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-znp5n" event={"ID":"22921ac9-5226-4222-861e-1412f2515c1f","Type":"ContainerStarted","Data":"ac65b5c56d0e09f795676f8ee3acb94d587bd2ad09cfb8ab67308b5a81bc2a24"} Jul 24 03:23:41.110323 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:41.110274 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-znp5n" podStartSLOduration=1.888912395 podStartE2EDuration="5.110261484s" totalImagesPullingTime="3.221349089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:23:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:23:41.109574732 +0000 UTC m=+318.385146085" watchObservedRunningTime="2026-07-24 03:23:41.110261484 +0000 UTC m=+318.385832834" Jul 24 03:23:45.142335 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:45.142296 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lpsxq"] Jul 24 03:23:45.145296 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:45.145279 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-lpsxq" Jul 24 03:23:45.147768 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:45.147743 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-h6tmr\"" Jul 24 03:23:45.156912 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:45.156891 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lpsxq"] Jul 24 03:23:45.207268 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:45.207247 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tk42p\" (UniqueName: \"kubernetes.io/projected/da65188e-5a99-4f6d-a501-c84625212be5-kube-api-access-tk42p\") pod \"cert-manager-55b4d954d5-lpsxq\" (UID: \"da65188e-5a99-4f6d-a501-c84625212be5\") " pod="cert-manager/cert-manager-55b4d954d5-lpsxq" Jul 24 03:23:45.207385 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:45.207319 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/da65188e-5a99-4f6d-a501-c84625212be5-bound-sa-token\") pod \"cert-manager-55b4d954d5-lpsxq\" (UID: \"da65188e-5a99-4f6d-a501-c84625212be5\") " pod="cert-manager/cert-manager-55b4d954d5-lpsxq" Jul 24 03:23:45.308437 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:45.308400 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/da65188e-5a99-4f6d-a501-c84625212be5-bound-sa-token\") pod \"cert-manager-55b4d954d5-lpsxq\" (UID: \"da65188e-5a99-4f6d-a501-c84625212be5\") " pod="cert-manager/cert-manager-55b4d954d5-lpsxq" Jul 24 03:23:45.308579 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:45.308464 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tk42p\" (UniqueName: \"kubernetes.io/projected/da65188e-5a99-4f6d-a501-c84625212be5-kube-api-access-tk42p\") pod \"cert-manager-55b4d954d5-lpsxq\" (UID: \"da65188e-5a99-4f6d-a501-c84625212be5\") " pod="cert-manager/cert-manager-55b4d954d5-lpsxq" Jul 24 03:23:45.317750 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:45.317731 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tk42p\" (UniqueName: \"kubernetes.io/projected/da65188e-5a99-4f6d-a501-c84625212be5-kube-api-access-tk42p\") pod \"cert-manager-55b4d954d5-lpsxq\" (UID: \"da65188e-5a99-4f6d-a501-c84625212be5\") " pod="cert-manager/cert-manager-55b4d954d5-lpsxq" Jul 24 03:23:45.317844 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:45.317730 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/da65188e-5a99-4f6d-a501-c84625212be5-bound-sa-token\") pod \"cert-manager-55b4d954d5-lpsxq\" (UID: \"da65188e-5a99-4f6d-a501-c84625212be5\") " pod="cert-manager/cert-manager-55b4d954d5-lpsxq" Jul 24 03:23:45.453600 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:45.453523 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-lpsxq" Jul 24 03:23:45.574991 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:45.574953 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lpsxq"] Jul 24 03:23:45.577650 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:23:45.577624 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podda65188e_5a99_4f6d_a501_c84625212be5.slice/crio-b25886e30b6c4c1a67dc7ebaa183ddc6c6890f32c7b111ee676ba537e5b26783 WatchSource:0}: Error finding container b25886e30b6c4c1a67dc7ebaa183ddc6c6890f32c7b111ee676ba537e5b26783: Status 404 returned error can't find the container with id b25886e30b6c4c1a67dc7ebaa183ddc6c6890f32c7b111ee676ba537e5b26783 Jul 24 03:23:46.120528 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:46.120487 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-lpsxq" event={"ID":"da65188e-5a99-4f6d-a501-c84625212be5","Type":"ContainerStarted","Data":"5ae04414d5b6d754135804f608406031f7b1667d496d4574539594c2135aa434"} Jul 24 03:23:46.120714 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:46.120537 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-lpsxq" event={"ID":"da65188e-5a99-4f6d-a501-c84625212be5","Type":"ContainerStarted","Data":"b25886e30b6c4c1a67dc7ebaa183ddc6c6890f32c7b111ee676ba537e5b26783"} Jul 24 03:23:46.137495 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:23:46.137451 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-lpsxq" podStartSLOduration=1.137437477 podStartE2EDuration="1.137437477s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:23:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:23:46.136595837 +0000 UTC m=+323.412167178" watchObservedRunningTime="2026-07-24 03:23:46.137437477 +0000 UTC m=+323.413008827" Jul 24 03:24:11.374058 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.374022 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c75c644cf-j7hn4"] Jul 24 03:24:11.377697 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.377656 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5c75c644cf-j7hn4" Jul 24 03:24:11.382496 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.382472 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 24 03:24:11.383571 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.383548 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 24 03:24:11.383865 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.383845 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 24 03:24:11.383947 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.383903 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 24 03:24:11.383947 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.383934 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-l74m9\"" Jul 24 03:24:11.397074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.397050 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c75c644cf-j7hn4"] Jul 24 03:24:11.512326 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.512295 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88mkn\" (UniqueName: \"kubernetes.io/projected/b56fa8eb-8fb8-4dea-a80d-5e9dee75941e-kube-api-access-88mkn\") pod \"opendatahub-operator-controller-manager-5c75c644cf-j7hn4\" (UID: \"b56fa8eb-8fb8-4dea-a80d-5e9dee75941e\") " pod="opendatahub/opendatahub-operator-controller-manager-5c75c644cf-j7hn4" Jul 24 03:24:11.512477 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.512336 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b56fa8eb-8fb8-4dea-a80d-5e9dee75941e-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c75c644cf-j7hn4\" (UID: \"b56fa8eb-8fb8-4dea-a80d-5e9dee75941e\") " pod="opendatahub/opendatahub-operator-controller-manager-5c75c644cf-j7hn4" Jul 24 03:24:11.512477 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.512358 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b56fa8eb-8fb8-4dea-a80d-5e9dee75941e-tmp\") pod \"opendatahub-operator-controller-manager-5c75c644cf-j7hn4\" (UID: \"b56fa8eb-8fb8-4dea-a80d-5e9dee75941e\") " pod="opendatahub/opendatahub-operator-controller-manager-5c75c644cf-j7hn4" Jul 24 03:24:11.512477 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.512455 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b56fa8eb-8fb8-4dea-a80d-5e9dee75941e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c75c644cf-j7hn4\" (UID: \"b56fa8eb-8fb8-4dea-a80d-5e9dee75941e\") " pod="opendatahub/opendatahub-operator-controller-manager-5c75c644cf-j7hn4" Jul 24 03:24:11.512578 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.512513 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b56fa8eb-8fb8-4dea-a80d-5e9dee75941e-manifests\") pod \"opendatahub-operator-controller-manager-5c75c644cf-j7hn4\" (UID: \"b56fa8eb-8fb8-4dea-a80d-5e9dee75941e\") " pod="opendatahub/opendatahub-operator-controller-manager-5c75c644cf-j7hn4" Jul 24 03:24:11.600087 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.600052 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-df5f88749-znv82"] Jul 24 03:24:11.603239 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.603217 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-df5f88749-znv82" Jul 24 03:24:11.606281 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.606252 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-8qxrp\"" Jul 24 03:24:11.606281 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.606277 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 24 03:24:11.606458 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.606281 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 24 03:24:11.606458 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.606452 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 24 03:24:11.606578 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.606532 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 24 03:24:11.606612 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.606573 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 24 03:24:11.613575 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.613556 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b56fa8eb-8fb8-4dea-a80d-5e9dee75941e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c75c644cf-j7hn4\" (UID: \"b56fa8eb-8fb8-4dea-a80d-5e9dee75941e\") " pod="opendatahub/opendatahub-operator-controller-manager-5c75c644cf-j7hn4" Jul 24 03:24:11.613679 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.613600 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b56fa8eb-8fb8-4dea-a80d-5e9dee75941e-manifests\") pod \"opendatahub-operator-controller-manager-5c75c644cf-j7hn4\" (UID: \"b56fa8eb-8fb8-4dea-a80d-5e9dee75941e\") " pod="opendatahub/opendatahub-operator-controller-manager-5c75c644cf-j7hn4" Jul 24 03:24:11.613679 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.613625 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88mkn\" (UniqueName: \"kubernetes.io/projected/b56fa8eb-8fb8-4dea-a80d-5e9dee75941e-kube-api-access-88mkn\") pod \"opendatahub-operator-controller-manager-5c75c644cf-j7hn4\" (UID: \"b56fa8eb-8fb8-4dea-a80d-5e9dee75941e\") " pod="opendatahub/opendatahub-operator-controller-manager-5c75c644cf-j7hn4" Jul 24 03:24:11.613679 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.613646 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b56fa8eb-8fb8-4dea-a80d-5e9dee75941e-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c75c644cf-j7hn4\" (UID: \"b56fa8eb-8fb8-4dea-a80d-5e9dee75941e\") " pod="opendatahub/opendatahub-operator-controller-manager-5c75c644cf-j7hn4" Jul 24 03:24:11.613803 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.613679 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b56fa8eb-8fb8-4dea-a80d-5e9dee75941e-tmp\") pod \"opendatahub-operator-controller-manager-5c75c644cf-j7hn4\" (UID: \"b56fa8eb-8fb8-4dea-a80d-5e9dee75941e\") " pod="opendatahub/opendatahub-operator-controller-manager-5c75c644cf-j7hn4" Jul 24 03:24:11.614045 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.614025 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b56fa8eb-8fb8-4dea-a80d-5e9dee75941e-manifests\") pod \"opendatahub-operator-controller-manager-5c75c644cf-j7hn4\" (UID: \"b56fa8eb-8fb8-4dea-a80d-5e9dee75941e\") " pod="opendatahub/opendatahub-operator-controller-manager-5c75c644cf-j7hn4" Jul 24 03:24:11.615861 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.615835 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b56fa8eb-8fb8-4dea-a80d-5e9dee75941e-tmp\") pod \"opendatahub-operator-controller-manager-5c75c644cf-j7hn4\" (UID: \"b56fa8eb-8fb8-4dea-a80d-5e9dee75941e\") " pod="opendatahub/opendatahub-operator-controller-manager-5c75c644cf-j7hn4" Jul 24 03:24:11.616040 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.616023 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b56fa8eb-8fb8-4dea-a80d-5e9dee75941e-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c75c644cf-j7hn4\" (UID: \"b56fa8eb-8fb8-4dea-a80d-5e9dee75941e\") " pod="opendatahub/opendatahub-operator-controller-manager-5c75c644cf-j7hn4" Jul 24 03:24:11.616099 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.616036 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b56fa8eb-8fb8-4dea-a80d-5e9dee75941e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c75c644cf-j7hn4\" (UID: \"b56fa8eb-8fb8-4dea-a80d-5e9dee75941e\") " pod="opendatahub/opendatahub-operator-controller-manager-5c75c644cf-j7hn4" Jul 24 03:24:11.629540 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.629485 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-df5f88749-znv82"] Jul 24 03:24:11.635320 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.635295 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88mkn\" (UniqueName: \"kubernetes.io/projected/b56fa8eb-8fb8-4dea-a80d-5e9dee75941e-kube-api-access-88mkn\") pod \"opendatahub-operator-controller-manager-5c75c644cf-j7hn4\" (UID: \"b56fa8eb-8fb8-4dea-a80d-5e9dee75941e\") " pod="opendatahub/opendatahub-operator-controller-manager-5c75c644cf-j7hn4" Jul 24 03:24:11.688915 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.688885 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5c75c644cf-j7hn4" Jul 24 03:24:11.714172 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.714144 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ada0295b-a707-4100-8d0f-9bf799c9fdc4-cert\") pod \"lws-controller-manager-df5f88749-znv82\" (UID: \"ada0295b-a707-4100-8d0f-9bf799c9fdc4\") " pod="openshift-lws-operator/lws-controller-manager-df5f88749-znv82" Jul 24 03:24:11.714303 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.714197 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzhpx\" (UniqueName: \"kubernetes.io/projected/ada0295b-a707-4100-8d0f-9bf799c9fdc4-kube-api-access-nzhpx\") pod \"lws-controller-manager-df5f88749-znv82\" (UID: \"ada0295b-a707-4100-8d0f-9bf799c9fdc4\") " pod="openshift-lws-operator/lws-controller-manager-df5f88749-znv82" Jul 24 03:24:11.714303 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.714224 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ada0295b-a707-4100-8d0f-9bf799c9fdc4-manager-config\") pod \"lws-controller-manager-df5f88749-znv82\" (UID: \"ada0295b-a707-4100-8d0f-9bf799c9fdc4\") " pod="openshift-lws-operator/lws-controller-manager-df5f88749-znv82" Jul 24 03:24:11.714409 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.714304 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ada0295b-a707-4100-8d0f-9bf799c9fdc4-metrics-cert\") pod \"lws-controller-manager-df5f88749-znv82\" (UID: \"ada0295b-a707-4100-8d0f-9bf799c9fdc4\") " pod="openshift-lws-operator/lws-controller-manager-df5f88749-znv82" Jul 24 03:24:11.815048 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.815019 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ada0295b-a707-4100-8d0f-9bf799c9fdc4-cert\") pod \"lws-controller-manager-df5f88749-znv82\" (UID: \"ada0295b-a707-4100-8d0f-9bf799c9fdc4\") " pod="openshift-lws-operator/lws-controller-manager-df5f88749-znv82" Jul 24 03:24:11.815207 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.815071 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzhpx\" (UniqueName: \"kubernetes.io/projected/ada0295b-a707-4100-8d0f-9bf799c9fdc4-kube-api-access-nzhpx\") pod \"lws-controller-manager-df5f88749-znv82\" (UID: \"ada0295b-a707-4100-8d0f-9bf799c9fdc4\") " pod="openshift-lws-operator/lws-controller-manager-df5f88749-znv82" Jul 24 03:24:11.815207 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.815100 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ada0295b-a707-4100-8d0f-9bf799c9fdc4-manager-config\") pod \"lws-controller-manager-df5f88749-znv82\" (UID: \"ada0295b-a707-4100-8d0f-9bf799c9fdc4\") " pod="openshift-lws-operator/lws-controller-manager-df5f88749-znv82" Jul 24 03:24:11.815207 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.815123 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ada0295b-a707-4100-8d0f-9bf799c9fdc4-metrics-cert\") pod \"lws-controller-manager-df5f88749-znv82\" (UID: \"ada0295b-a707-4100-8d0f-9bf799c9fdc4\") " pod="openshift-lws-operator/lws-controller-manager-df5f88749-znv82" Jul 24 03:24:11.815838 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.815815 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ada0295b-a707-4100-8d0f-9bf799c9fdc4-manager-config\") pod \"lws-controller-manager-df5f88749-znv82\" (UID: \"ada0295b-a707-4100-8d0f-9bf799c9fdc4\") " pod="openshift-lws-operator/lws-controller-manager-df5f88749-znv82" Jul 24 03:24:11.817393 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.817375 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ada0295b-a707-4100-8d0f-9bf799c9fdc4-metrics-cert\") pod \"lws-controller-manager-df5f88749-znv82\" (UID: \"ada0295b-a707-4100-8d0f-9bf799c9fdc4\") " pod="openshift-lws-operator/lws-controller-manager-df5f88749-znv82" Jul 24 03:24:11.817497 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.817481 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ada0295b-a707-4100-8d0f-9bf799c9fdc4-cert\") pod \"lws-controller-manager-df5f88749-znv82\" (UID: \"ada0295b-a707-4100-8d0f-9bf799c9fdc4\") " pod="openshift-lws-operator/lws-controller-manager-df5f88749-znv82" Jul 24 03:24:11.833690 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.833651 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzhpx\" (UniqueName: \"kubernetes.io/projected/ada0295b-a707-4100-8d0f-9bf799c9fdc4-kube-api-access-nzhpx\") pod \"lws-controller-manager-df5f88749-znv82\" (UID: \"ada0295b-a707-4100-8d0f-9bf799c9fdc4\") " pod="openshift-lws-operator/lws-controller-manager-df5f88749-znv82" Jul 24 03:24:11.853468 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.851655 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c75c644cf-j7hn4"] Jul 24 03:24:11.857691 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:24:11.857650 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb56fa8eb_8fb8_4dea_a80d_5e9dee75941e.slice/crio-a75d9110cf1c83cac774b67488df9d14432c80e12d095a44b1be2699ca8d0077 WatchSource:0}: Error finding container a75d9110cf1c83cac774b67488df9d14432c80e12d095a44b1be2699ca8d0077: Status 404 returned error can't find the container with id a75d9110cf1c83cac774b67488df9d14432c80e12d095a44b1be2699ca8d0077 Jul 24 03:24:11.912319 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:11.912257 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-df5f88749-znv82" Jul 24 03:24:12.047547 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:12.047513 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-df5f88749-znv82"] Jul 24 03:24:12.050085 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:24:12.050052 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podada0295b_a707_4100_8d0f_9bf799c9fdc4.slice/crio-1f51f462cd1c6e34af79c9efe8d6e8f676cb085aecc198c431ab0caef33ba019 WatchSource:0}: Error finding container 1f51f462cd1c6e34af79c9efe8d6e8f676cb085aecc198c431ab0caef33ba019: Status 404 returned error can't find the container with id 1f51f462cd1c6e34af79c9efe8d6e8f676cb085aecc198c431ab0caef33ba019 Jul 24 03:24:12.226308 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:12.226216 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-df5f88749-znv82" event={"ID":"ada0295b-a707-4100-8d0f-9bf799c9fdc4","Type":"ContainerStarted","Data":"1f51f462cd1c6e34af79c9efe8d6e8f676cb085aecc198c431ab0caef33ba019"} Jul 24 03:24:12.227338 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:12.227315 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c75c644cf-j7hn4" event={"ID":"b56fa8eb-8fb8-4dea-a80d-5e9dee75941e","Type":"ContainerStarted","Data":"a75d9110cf1c83cac774b67488df9d14432c80e12d095a44b1be2699ca8d0077"} Jul 24 03:24:16.250276 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:16.250233 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-df5f88749-znv82" event={"ID":"ada0295b-a707-4100-8d0f-9bf799c9fdc4","Type":"ContainerStarted","Data":"25910553243f238c5ccec3ca7c422a3d8a7a9ecbb2b90438a9abcf00c6c907ab"} Jul 24 03:24:16.250735 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:16.250300 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-df5f88749-znv82" Jul 24 03:24:16.251692 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:16.251657 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c75c644cf-j7hn4" event={"ID":"b56fa8eb-8fb8-4dea-a80d-5e9dee75941e","Type":"ContainerStarted","Data":"6cc03059ef88f848b9b05b84a7ccdec3d3f6de61fe567fa3daf01deb91f6c726"} Jul 24 03:24:16.271803 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:16.271762 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-df5f88749-znv82" podStartSLOduration=1.248974326 podStartE2EDuration="5.271750654s" totalImagesPullingTime="4.022776328s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:24:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:24:16.270137403 +0000 UTC m=+353.545708754" watchObservedRunningTime="2026-07-24 03:24:16.271750654 +0000 UTC m=+353.547322071" Jul 24 03:24:17.256925 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:17.256888 2579 generic.go:356] "Generic (PLEG): container finished" podID="b56fa8eb-8fb8-4dea-a80d-5e9dee75941e" containerID="6cc03059ef88f848b9b05b84a7ccdec3d3f6de61fe567fa3daf01deb91f6c726" exitCode=0 Jul 24 03:24:17.257322 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:17.256975 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c75c644cf-j7hn4" event={"ID":"b56fa8eb-8fb8-4dea-a80d-5e9dee75941e","Type":"ContainerDied","Data":"6cc03059ef88f848b9b05b84a7ccdec3d3f6de61fe567fa3daf01deb91f6c726"} Jul 24 03:24:18.262659 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:18.262626 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c75c644cf-j7hn4" event={"ID":"b56fa8eb-8fb8-4dea-a80d-5e9dee75941e","Type":"ContainerStarted","Data":"f6d2f6e95b2fd5bddcab1dca5cb6d6d55a89d679c98373cc6cca1155bf10200d"} Jul 24 03:24:18.263058 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:18.262780 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5c75c644cf-j7hn4" Jul 24 03:24:18.285687 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:18.285617 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5c75c644cf-j7hn4" podStartSLOduration=1.723810087 podStartE2EDuration="7.285599447s" totalImagesPullingTime="4.566696395s" totalInitContainerRuntime="995.092965ms" isStatefulPod=false podCreationTimestamp="2026-07-24 03:24:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:24:18.284202904 +0000 UTC m=+355.559774255" watchObservedRunningTime="2026-07-24 03:24:18.285599447 +0000 UTC m=+355.561170797" Jul 24 03:24:27.259982 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:27.259952 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-df5f88749-znv82" Jul 24 03:24:29.271162 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:29.271135 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5c75c644cf-j7hn4" Jul 24 03:24:31.628086 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:31.628054 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7ccbc5c95c-n9qf9"] Jul 24 03:24:31.636881 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:31.636863 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7ccbc5c95c-n9qf9" Jul 24 03:24:31.639239 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:31.639217 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 24 03:24:31.639365 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:31.639271 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 24 03:24:31.639550 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:31.639533 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-kqqr6\"" Jul 24 03:24:31.639818 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:31.639804 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 24 03:24:31.647065 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:31.647048 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 24 03:24:31.653841 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:31.653821 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7ccbc5c95c-n9qf9"] Jul 24 03:24:31.671697 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:31.671660 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a84843d1-19b7-468e-ac6f-f925ed893a36-tmp\") pod \"kube-auth-proxy-7ccbc5c95c-n9qf9\" (UID: \"a84843d1-19b7-468e-ac6f-f925ed893a36\") " pod="openshift-ingress/kube-auth-proxy-7ccbc5c95c-n9qf9" Jul 24 03:24:31.671790 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:31.671740 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8dfh\" (UniqueName: \"kubernetes.io/projected/a84843d1-19b7-468e-ac6f-f925ed893a36-kube-api-access-p8dfh\") pod \"kube-auth-proxy-7ccbc5c95c-n9qf9\" (UID: \"a84843d1-19b7-468e-ac6f-f925ed893a36\") " pod="openshift-ingress/kube-auth-proxy-7ccbc5c95c-n9qf9" Jul 24 03:24:31.671838 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:31.671795 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a84843d1-19b7-468e-ac6f-f925ed893a36-tls-certs\") pod \"kube-auth-proxy-7ccbc5c95c-n9qf9\" (UID: \"a84843d1-19b7-468e-ac6f-f925ed893a36\") " pod="openshift-ingress/kube-auth-proxy-7ccbc5c95c-n9qf9" Jul 24 03:24:31.772384 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:31.772353 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a84843d1-19b7-468e-ac6f-f925ed893a36-tmp\") pod \"kube-auth-proxy-7ccbc5c95c-n9qf9\" (UID: \"a84843d1-19b7-468e-ac6f-f925ed893a36\") " pod="openshift-ingress/kube-auth-proxy-7ccbc5c95c-n9qf9" Jul 24 03:24:31.772549 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:31.772397 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8dfh\" (UniqueName: \"kubernetes.io/projected/a84843d1-19b7-468e-ac6f-f925ed893a36-kube-api-access-p8dfh\") pod \"kube-auth-proxy-7ccbc5c95c-n9qf9\" (UID: \"a84843d1-19b7-468e-ac6f-f925ed893a36\") " pod="openshift-ingress/kube-auth-proxy-7ccbc5c95c-n9qf9" Jul 24 03:24:31.772549 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:31.772429 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a84843d1-19b7-468e-ac6f-f925ed893a36-tls-certs\") pod \"kube-auth-proxy-7ccbc5c95c-n9qf9\" (UID: \"a84843d1-19b7-468e-ac6f-f925ed893a36\") " pod="openshift-ingress/kube-auth-proxy-7ccbc5c95c-n9qf9" Jul 24 03:24:31.772628 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:24:31.772571 2579 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 24 03:24:31.772723 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:24:31.772652 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a84843d1-19b7-468e-ac6f-f925ed893a36-tls-certs podName:a84843d1-19b7-468e-ac6f-f925ed893a36 nodeName:}" failed. No retries permitted until 2026-07-24 03:24:32.272631447 +0000 UTC m=+369.548202794 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/a84843d1-19b7-468e-ac6f-f925ed893a36-tls-certs") pod "kube-auth-proxy-7ccbc5c95c-n9qf9" (UID: "a84843d1-19b7-468e-ac6f-f925ed893a36") : secret "kube-auth-proxy-tls" not found Jul 24 03:24:31.774596 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:31.774573 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a84843d1-19b7-468e-ac6f-f925ed893a36-tmp\") pod \"kube-auth-proxy-7ccbc5c95c-n9qf9\" (UID: \"a84843d1-19b7-468e-ac6f-f925ed893a36\") " pod="openshift-ingress/kube-auth-proxy-7ccbc5c95c-n9qf9" Jul 24 03:24:31.802161 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:31.802131 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8dfh\" (UniqueName: \"kubernetes.io/projected/a84843d1-19b7-468e-ac6f-f925ed893a36-kube-api-access-p8dfh\") pod \"kube-auth-proxy-7ccbc5c95c-n9qf9\" (UID: \"a84843d1-19b7-468e-ac6f-f925ed893a36\") " pod="openshift-ingress/kube-auth-proxy-7ccbc5c95c-n9qf9" Jul 24 03:24:32.277964 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:32.277931 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a84843d1-19b7-468e-ac6f-f925ed893a36-tls-certs\") pod \"kube-auth-proxy-7ccbc5c95c-n9qf9\" (UID: \"a84843d1-19b7-468e-ac6f-f925ed893a36\") " pod="openshift-ingress/kube-auth-proxy-7ccbc5c95c-n9qf9" Jul 24 03:24:32.278137 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:24:32.278110 2579 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 24 03:24:32.278203 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:24:32.278191 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a84843d1-19b7-468e-ac6f-f925ed893a36-tls-certs podName:a84843d1-19b7-468e-ac6f-f925ed893a36 nodeName:}" failed. No retries permitted until 2026-07-24 03:24:33.278171237 +0000 UTC m=+370.553742591 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/a84843d1-19b7-468e-ac6f-f925ed893a36-tls-certs") pod "kube-auth-proxy-7ccbc5c95c-n9qf9" (UID: "a84843d1-19b7-468e-ac6f-f925ed893a36") : secret "kube-auth-proxy-tls" not found Jul 24 03:24:33.284645 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:33.284610 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a84843d1-19b7-468e-ac6f-f925ed893a36-tls-certs\") pod \"kube-auth-proxy-7ccbc5c95c-n9qf9\" (UID: \"a84843d1-19b7-468e-ac6f-f925ed893a36\") " pod="openshift-ingress/kube-auth-proxy-7ccbc5c95c-n9qf9" Jul 24 03:24:33.286892 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:33.286870 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a84843d1-19b7-468e-ac6f-f925ed893a36-tls-certs\") pod \"kube-auth-proxy-7ccbc5c95c-n9qf9\" (UID: \"a84843d1-19b7-468e-ac6f-f925ed893a36\") " pod="openshift-ingress/kube-auth-proxy-7ccbc5c95c-n9qf9" Jul 24 03:24:33.446778 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:33.446750 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7ccbc5c95c-n9qf9" Jul 24 03:24:33.583255 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:33.583224 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7ccbc5c95c-n9qf9"] Jul 24 03:24:33.585972 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:24:33.585944 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda84843d1_19b7_468e_ac6f_f925ed893a36.slice/crio-f318fbe6ad86bb316e37c6414762abaf0b555e9b17c5ce7a05afc20b70565949 WatchSource:0}: Error finding container f318fbe6ad86bb316e37c6414762abaf0b555e9b17c5ce7a05afc20b70565949: Status 404 returned error can't find the container with id f318fbe6ad86bb316e37c6414762abaf0b555e9b17c5ce7a05afc20b70565949 Jul 24 03:24:34.330445 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:34.330409 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7ccbc5c95c-n9qf9" event={"ID":"a84843d1-19b7-468e-ac6f-f925ed893a36","Type":"ContainerStarted","Data":"f318fbe6ad86bb316e37c6414762abaf0b555e9b17c5ce7a05afc20b70565949"} Jul 24 03:24:36.340202 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:36.340162 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7ccbc5c95c-n9qf9" event={"ID":"a84843d1-19b7-468e-ac6f-f925ed893a36","Type":"ContainerStarted","Data":"e588770c1da022d625430fe8256c14779b03488860d351fb9d5ccf5341a99fd7"} Jul 24 03:24:36.356463 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:24:36.356415 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7ccbc5c95c-n9qf9" podStartSLOduration=3.332757423 podStartE2EDuration="5.356399942s" totalImagesPullingTime="2.023642519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:24:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:24:36.355977511 +0000 UTC m=+373.631548861" watchObservedRunningTime="2026-07-24 03:24:36.356399942 +0000 UTC m=+373.631971296" Jul 24 03:25:57.084910 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:25:57.084872 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-66967c796d-tq4w5"] Jul 24 03:25:57.088277 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:25:57.088260 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66967c796d-tq4w5" Jul 24 03:25:57.091643 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:25:57.091621 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 24 03:25:57.091845 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:25:57.091825 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 24 03:25:57.091909 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:25:57.091890 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 24 03:25:57.091909 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:25:57.091903 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 24 03:25:57.092276 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:25:57.092255 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 24 03:25:57.092771 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:25:57.092757 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-mvmn5\"" Jul 24 03:25:57.103344 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:25:57.103318 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 24 03:25:57.104906 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:25:57.104885 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66967c796d-tq4w5"] Jul 24 03:25:57.145815 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:25:57.145793 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bde7a761-9336-4804-9891-9a8c0ca5ea3c-console-oauth-config\") pod \"console-66967c796d-tq4w5\" (UID: \"bde7a761-9336-4804-9891-9a8c0ca5ea3c\") " pod="openshift-console/console-66967c796d-tq4w5" Jul 24 03:25:57.145933 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:25:57.145818 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bde7a761-9336-4804-9891-9a8c0ca5ea3c-trusted-ca-bundle\") pod \"console-66967c796d-tq4w5\" (UID: \"bde7a761-9336-4804-9891-9a8c0ca5ea3c\") " pod="openshift-console/console-66967c796d-tq4w5" Jul 24 03:25:57.145933 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:25:57.145835 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bde7a761-9336-4804-9891-9a8c0ca5ea3c-oauth-serving-cert\") pod \"console-66967c796d-tq4w5\" (UID: \"bde7a761-9336-4804-9891-9a8c0ca5ea3c\") " pod="openshift-console/console-66967c796d-tq4w5" Jul 24 03:25:57.145933 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:25:57.145850 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bde7a761-9336-4804-9891-9a8c0ca5ea3c-console-serving-cert\") pod \"console-66967c796d-tq4w5\" (UID: \"bde7a761-9336-4804-9891-9a8c0ca5ea3c\") " pod="openshift-console/console-66967c796d-tq4w5" Jul 24 03:25:57.146105 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:25:57.145966 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bde7a761-9336-4804-9891-9a8c0ca5ea3c-service-ca\") pod \"console-66967c796d-tq4w5\" (UID: \"bde7a761-9336-4804-9891-9a8c0ca5ea3c\") " pod="openshift-console/console-66967c796d-tq4w5" Jul 24 03:25:57.146105 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:25:57.146047 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bde7a761-9336-4804-9891-9a8c0ca5ea3c-console-config\") pod \"console-66967c796d-tq4w5\" (UID: \"bde7a761-9336-4804-9891-9a8c0ca5ea3c\") " pod="openshift-console/console-66967c796d-tq4w5" Jul 24 03:25:57.146229 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:25:57.146136 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cq8zs\" (UniqueName: \"kubernetes.io/projected/bde7a761-9336-4804-9891-9a8c0ca5ea3c-kube-api-access-cq8zs\") pod \"console-66967c796d-tq4w5\" (UID: \"bde7a761-9336-4804-9891-9a8c0ca5ea3c\") " pod="openshift-console/console-66967c796d-tq4w5" Jul 24 03:25:57.247239 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:25:57.247212 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bde7a761-9336-4804-9891-9a8c0ca5ea3c-console-config\") pod \"console-66967c796d-tq4w5\" (UID: \"bde7a761-9336-4804-9891-9a8c0ca5ea3c\") " pod="openshift-console/console-66967c796d-tq4w5" Jul 24 03:25:57.247378 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:25:57.247257 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cq8zs\" (UniqueName: \"kubernetes.io/projected/bde7a761-9336-4804-9891-9a8c0ca5ea3c-kube-api-access-cq8zs\") pod \"console-66967c796d-tq4w5\" (UID: \"bde7a761-9336-4804-9891-9a8c0ca5ea3c\") " pod="openshift-console/console-66967c796d-tq4w5" Jul 24 03:25:57.247378 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:25:57.247273 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bde7a761-9336-4804-9891-9a8c0ca5ea3c-console-oauth-config\") pod \"console-66967c796d-tq4w5\" (UID: \"bde7a761-9336-4804-9891-9a8c0ca5ea3c\") " pod="openshift-console/console-66967c796d-tq4w5" Jul 24 03:25:57.247490 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:25:57.247382 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bde7a761-9336-4804-9891-9a8c0ca5ea3c-trusted-ca-bundle\") pod \"console-66967c796d-tq4w5\" (UID: \"bde7a761-9336-4804-9891-9a8c0ca5ea3c\") " pod="openshift-console/console-66967c796d-tq4w5" Jul 24 03:25:57.247490 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:25:57.247412 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bde7a761-9336-4804-9891-9a8c0ca5ea3c-oauth-serving-cert\") pod \"console-66967c796d-tq4w5\" (UID: \"bde7a761-9336-4804-9891-9a8c0ca5ea3c\") " pod="openshift-console/console-66967c796d-tq4w5" Jul 24 03:25:57.247490 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:25:57.247431 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bde7a761-9336-4804-9891-9a8c0ca5ea3c-console-serving-cert\") pod \"console-66967c796d-tq4w5\" (UID: \"bde7a761-9336-4804-9891-9a8c0ca5ea3c\") " pod="openshift-console/console-66967c796d-tq4w5" Jul 24 03:25:57.247490 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:25:57.247469 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bde7a761-9336-4804-9891-9a8c0ca5ea3c-service-ca\") pod \"console-66967c796d-tq4w5\" (UID: \"bde7a761-9336-4804-9891-9a8c0ca5ea3c\") " pod="openshift-console/console-66967c796d-tq4w5" Jul 24 03:25:57.248081 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:25:57.248051 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bde7a761-9336-4804-9891-9a8c0ca5ea3c-oauth-serving-cert\") pod \"console-66967c796d-tq4w5\" (UID: \"bde7a761-9336-4804-9891-9a8c0ca5ea3c\") " pod="openshift-console/console-66967c796d-tq4w5" Jul 24 03:25:57.248081 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:25:57.248064 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bde7a761-9336-4804-9891-9a8c0ca5ea3c-console-config\") pod \"console-66967c796d-tq4w5\" (UID: \"bde7a761-9336-4804-9891-9a8c0ca5ea3c\") " pod="openshift-console/console-66967c796d-tq4w5" Jul 24 03:25:57.248226 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:25:57.248211 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bde7a761-9336-4804-9891-9a8c0ca5ea3c-service-ca\") pod \"console-66967c796d-tq4w5\" (UID: \"bde7a761-9336-4804-9891-9a8c0ca5ea3c\") " pod="openshift-console/console-66967c796d-tq4w5" Jul 24 03:25:57.248346 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:25:57.248323 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bde7a761-9336-4804-9891-9a8c0ca5ea3c-trusted-ca-bundle\") pod \"console-66967c796d-tq4w5\" (UID: \"bde7a761-9336-4804-9891-9a8c0ca5ea3c\") " pod="openshift-console/console-66967c796d-tq4w5" Jul 24 03:25:57.249819 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:25:57.249789 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bde7a761-9336-4804-9891-9a8c0ca5ea3c-console-oauth-config\") pod \"console-66967c796d-tq4w5\" (UID: \"bde7a761-9336-4804-9891-9a8c0ca5ea3c\") " pod="openshift-console/console-66967c796d-tq4w5" Jul 24 03:25:57.249962 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:25:57.249942 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bde7a761-9336-4804-9891-9a8c0ca5ea3c-console-serving-cert\") pod \"console-66967c796d-tq4w5\" (UID: \"bde7a761-9336-4804-9891-9a8c0ca5ea3c\") " pod="openshift-console/console-66967c796d-tq4w5" Jul 24 03:25:57.255176 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:25:57.255160 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cq8zs\" (UniqueName: \"kubernetes.io/projected/bde7a761-9336-4804-9891-9a8c0ca5ea3c-kube-api-access-cq8zs\") pod \"console-66967c796d-tq4w5\" (UID: \"bde7a761-9336-4804-9891-9a8c0ca5ea3c\") " pod="openshift-console/console-66967c796d-tq4w5" Jul 24 03:25:57.402792 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:25:57.402715 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66967c796d-tq4w5" Jul 24 03:25:57.530361 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:25:57.530338 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66967c796d-tq4w5"] Jul 24 03:25:57.532937 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:25:57.532909 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbde7a761_9336_4804_9891_9a8c0ca5ea3c.slice/crio-c31bf5612c82fe2679dadfa8ad6ed1d6b09e222818c246bcf04b6bfdc3df7c41 WatchSource:0}: Error finding container c31bf5612c82fe2679dadfa8ad6ed1d6b09e222818c246bcf04b6bfdc3df7c41: Status 404 returned error can't find the container with id c31bf5612c82fe2679dadfa8ad6ed1d6b09e222818c246bcf04b6bfdc3df7c41 Jul 24 03:25:57.682100 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:25:57.682019 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66967c796d-tq4w5" event={"ID":"bde7a761-9336-4804-9891-9a8c0ca5ea3c","Type":"ContainerStarted","Data":"2dfd83e1473f87d3d1f225f9dc5b509124fc104bffd93d614f4100507d41fdba"} Jul 24 03:25:57.682100 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:25:57.682071 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66967c796d-tq4w5" event={"ID":"bde7a761-9336-4804-9891-9a8c0ca5ea3c","Type":"ContainerStarted","Data":"c31bf5612c82fe2679dadfa8ad6ed1d6b09e222818c246bcf04b6bfdc3df7c41"} Jul 24 03:25:57.709857 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:25:57.709809 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-66967c796d-tq4w5" podStartSLOduration=0.709794641 podStartE2EDuration="709.794641ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:25:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:25:57.709183016 +0000 UTC m=+454.984754366" watchObservedRunningTime="2026-07-24 03:25:57.709794641 +0000 UTC m=+454.985365991" Jul 24 03:26:07.403280 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:07.403235 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66967c796d-tq4w5" Jul 24 03:26:07.403718 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:07.403302 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66967c796d-tq4w5" Jul 24 03:26:07.408042 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:07.408020 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-66967c796d-tq4w5" Jul 24 03:26:07.725554 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:07.725486 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-66967c796d-tq4w5" Jul 24 03:26:17.569075 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:17.569033 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-vkx42"] Jul 24 03:26:17.572303 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:17.572280 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-vkx42"] Jul 24 03:26:17.572435 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:17.572373 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vkx42" Jul 24 03:26:17.593388 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:17.593366 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-9t75r\"" Jul 24 03:26:17.595441 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:17.595419 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 24 03:26:17.601156 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:17.601135 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 24 03:26:17.601156 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:17.601153 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 24 03:26:17.601321 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:17.601223 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 24 03:26:17.721813 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:17.721787 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jdvl\" (UniqueName: \"kubernetes.io/projected/9c4703c1-46e3-4cfc-a3d2-9d9710fae811-kube-api-access-9jdvl\") pod \"kuadrant-console-plugin-66b5774985-vkx42\" (UID: \"9c4703c1-46e3-4cfc-a3d2-9d9710fae811\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vkx42" Jul 24 03:26:17.721919 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:17.721819 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9c4703c1-46e3-4cfc-a3d2-9d9710fae811-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-vkx42\" (UID: \"9c4703c1-46e3-4cfc-a3d2-9d9710fae811\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vkx42" Jul 24 03:26:17.721977 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:17.721957 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9c4703c1-46e3-4cfc-a3d2-9d9710fae811-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-vkx42\" (UID: \"9c4703c1-46e3-4cfc-a3d2-9d9710fae811\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vkx42" Jul 24 03:26:17.823232 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:17.823204 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9c4703c1-46e3-4cfc-a3d2-9d9710fae811-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-vkx42\" (UID: \"9c4703c1-46e3-4cfc-a3d2-9d9710fae811\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vkx42" Jul 24 03:26:17.823378 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:17.823243 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jdvl\" (UniqueName: \"kubernetes.io/projected/9c4703c1-46e3-4cfc-a3d2-9d9710fae811-kube-api-access-9jdvl\") pod \"kuadrant-console-plugin-66b5774985-vkx42\" (UID: \"9c4703c1-46e3-4cfc-a3d2-9d9710fae811\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vkx42" Jul 24 03:26:17.823378 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:17.823263 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9c4703c1-46e3-4cfc-a3d2-9d9710fae811-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-vkx42\" (UID: \"9c4703c1-46e3-4cfc-a3d2-9d9710fae811\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vkx42" Jul 24 03:26:17.824098 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:17.824073 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9c4703c1-46e3-4cfc-a3d2-9d9710fae811-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-vkx42\" (UID: \"9c4703c1-46e3-4cfc-a3d2-9d9710fae811\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vkx42" Jul 24 03:26:17.825530 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:17.825512 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9c4703c1-46e3-4cfc-a3d2-9d9710fae811-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-vkx42\" (UID: \"9c4703c1-46e3-4cfc-a3d2-9d9710fae811\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vkx42" Jul 24 03:26:17.843632 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:17.843609 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jdvl\" (UniqueName: \"kubernetes.io/projected/9c4703c1-46e3-4cfc-a3d2-9d9710fae811-kube-api-access-9jdvl\") pod \"kuadrant-console-plugin-66b5774985-vkx42\" (UID: \"9c4703c1-46e3-4cfc-a3d2-9d9710fae811\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vkx42" Jul 24 03:26:17.882035 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:17.882017 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vkx42" Jul 24 03:26:18.060400 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:18.060373 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-vkx42"] Jul 24 03:26:18.063146 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:26:18.063121 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9c4703c1_46e3_4cfc_a3d2_9d9710fae811.slice/crio-b628025dc689f2e9ba47546169f9bc9d7d323a12df1db89036806b27f0ded701 WatchSource:0}: Error finding container b628025dc689f2e9ba47546169f9bc9d7d323a12df1db89036806b27f0ded701: Status 404 returned error can't find the container with id b628025dc689f2e9ba47546169f9bc9d7d323a12df1db89036806b27f0ded701 Jul 24 03:26:18.766894 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:18.766858 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vkx42" event={"ID":"9c4703c1-46e3-4cfc-a3d2-9d9710fae811","Type":"ContainerStarted","Data":"b628025dc689f2e9ba47546169f9bc9d7d323a12df1db89036806b27f0ded701"} Jul 24 03:26:28.432275 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:28.432237 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gmb7f"] Jul 24 03:26:28.488501 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:28.488470 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gmb7f"] Jul 24 03:26:28.488690 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:28.488566 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gmb7f" Jul 24 03:26:28.492783 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:28.492545 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-769mn\"" Jul 24 03:26:28.521915 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:28.521878 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58wzf\" (UniqueName: \"kubernetes.io/projected/a341ebec-2e2f-4178-86ee-17ecf3f5613f-kube-api-access-58wzf\") pod \"kuadrant-operator-controller-manager-86878c769f-gmb7f\" (UID: \"a341ebec-2e2f-4178-86ee-17ecf3f5613f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gmb7f" Jul 24 03:26:28.522041 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:28.521951 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a341ebec-2e2f-4178-86ee-17ecf3f5613f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gmb7f\" (UID: \"a341ebec-2e2f-4178-86ee-17ecf3f5613f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gmb7f" Jul 24 03:26:28.622730 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:28.622659 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a341ebec-2e2f-4178-86ee-17ecf3f5613f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gmb7f\" (UID: \"a341ebec-2e2f-4178-86ee-17ecf3f5613f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gmb7f" Jul 24 03:26:28.622920 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:28.622797 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58wzf\" (UniqueName: \"kubernetes.io/projected/a341ebec-2e2f-4178-86ee-17ecf3f5613f-kube-api-access-58wzf\") pod \"kuadrant-operator-controller-manager-86878c769f-gmb7f\" (UID: \"a341ebec-2e2f-4178-86ee-17ecf3f5613f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gmb7f" Jul 24 03:26:28.623029 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:28.623006 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a341ebec-2e2f-4178-86ee-17ecf3f5613f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gmb7f\" (UID: \"a341ebec-2e2f-4178-86ee-17ecf3f5613f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gmb7f" Jul 24 03:26:28.632774 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:28.632746 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58wzf\" (UniqueName: \"kubernetes.io/projected/a341ebec-2e2f-4178-86ee-17ecf3f5613f-kube-api-access-58wzf\") pod \"kuadrant-operator-controller-manager-86878c769f-gmb7f\" (UID: \"a341ebec-2e2f-4178-86ee-17ecf3f5613f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gmb7f" Jul 24 03:26:28.800815 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:28.800731 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gmb7f" Jul 24 03:26:38.906411 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:38.906344 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gmb7f"] Jul 24 03:26:38.908454 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:26:38.908429 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda341ebec_2e2f_4178_86ee_17ecf3f5613f.slice/crio-c60a4796e9968ff3fdba942f1611bdb18494f1c6efc652b16e0bcc43371a66be WatchSource:0}: Error finding container c60a4796e9968ff3fdba942f1611bdb18494f1c6efc652b16e0bcc43371a66be: Status 404 returned error can't find the container with id c60a4796e9968ff3fdba942f1611bdb18494f1c6efc652b16e0bcc43371a66be Jul 24 03:26:39.874250 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:39.874218 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gmb7f" event={"ID":"a341ebec-2e2f-4178-86ee-17ecf3f5613f","Type":"ContainerStarted","Data":"c60a4796e9968ff3fdba942f1611bdb18494f1c6efc652b16e0bcc43371a66be"} Jul 24 03:26:39.875612 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:39.875585 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vkx42" event={"ID":"9c4703c1-46e3-4cfc-a3d2-9d9710fae811","Type":"ContainerStarted","Data":"0701c3af678235ec2e9e5dc240b4aeba1a37e5a7027aeab7a743ea913e7f3342"} Jul 24 03:26:39.896310 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:39.896269 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vkx42" podStartSLOduration=2.113329019 podStartE2EDuration="22.896253088s" totalImagesPullingTime="20.782924069s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:26:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:26:39.89338174 +0000 UTC m=+497.168953089" watchObservedRunningTime="2026-07-24 03:26:39.896253088 +0000 UTC m=+497.171824465" Jul 24 03:26:44.901223 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:44.901180 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gmb7f" event={"ID":"a341ebec-2e2f-4178-86ee-17ecf3f5613f","Type":"ContainerStarted","Data":"52dfabd4ff9435dd76f50cea1cd32b94e21041ce5d486b5d1b7df7a8fac80501"} Jul 24 03:26:44.901612 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:44.901266 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gmb7f" Jul 24 03:26:44.942965 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:44.942913 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gmb7f" podStartSLOduration=11.60837374 podStartE2EDuration="16.942896997s" totalImagesPullingTime="5.334523257s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:26:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:26:44.942193929 +0000 UTC m=+502.217765280" watchObservedRunningTime="2026-07-24 03:26:44.942896997 +0000 UTC m=+502.218468346" Jul 24 03:26:55.907290 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:26:55.907259 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gmb7f" Jul 24 03:27:18.275655 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:18.275619 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-crlwn"] Jul 24 03:27:18.277796 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:18.277781 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-crlwn" Jul 24 03:27:18.279544 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:18.279526 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-jdbv2\"" Jul 24 03:27:18.284891 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:18.284518 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-crlwn"] Jul 24 03:27:18.352601 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:18.352566 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkhcf\" (UniqueName: \"kubernetes.io/projected/7160b880-9e04-45e9-a5d4-f47c2c6d645a-kube-api-access-wkhcf\") pod \"authorino-58cd955675-crlwn\" (UID: \"7160b880-9e04-45e9-a5d4-f47c2c6d645a\") " pod="kuadrant-system/authorino-58cd955675-crlwn" Jul 24 03:27:18.453441 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:18.453405 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkhcf\" (UniqueName: \"kubernetes.io/projected/7160b880-9e04-45e9-a5d4-f47c2c6d645a-kube-api-access-wkhcf\") pod \"authorino-58cd955675-crlwn\" (UID: \"7160b880-9e04-45e9-a5d4-f47c2c6d645a\") " pod="kuadrant-system/authorino-58cd955675-crlwn" Jul 24 03:27:18.460929 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:18.460895 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkhcf\" (UniqueName: \"kubernetes.io/projected/7160b880-9e04-45e9-a5d4-f47c2c6d645a-kube-api-access-wkhcf\") pod \"authorino-58cd955675-crlwn\" (UID: \"7160b880-9e04-45e9-a5d4-f47c2c6d645a\") " pod="kuadrant-system/authorino-58cd955675-crlwn" Jul 24 03:27:18.587830 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:18.587802 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-crlwn" Jul 24 03:27:18.711158 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:18.711134 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-crlwn"] Jul 24 03:27:18.713507 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:27:18.713482 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7160b880_9e04_45e9_a5d4_f47c2c6d645a.slice/crio-5f8cc47296a0a95d3706988d1ea1bfedb6d312f32e4358beffd158e4c3e6cf58 WatchSource:0}: Error finding container 5f8cc47296a0a95d3706988d1ea1bfedb6d312f32e4358beffd158e4c3e6cf58: Status 404 returned error can't find the container with id 5f8cc47296a0a95d3706988d1ea1bfedb6d312f32e4358beffd158e4c3e6cf58 Jul 24 03:27:19.046347 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:19.046265 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-crlwn" event={"ID":"7160b880-9e04-45e9-a5d4-f47c2c6d645a","Type":"ContainerStarted","Data":"5f8cc47296a0a95d3706988d1ea1bfedb6d312f32e4358beffd158e4c3e6cf58"} Jul 24 03:27:22.062342 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:22.062252 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-crlwn" event={"ID":"7160b880-9e04-45e9-a5d4-f47c2c6d645a","Type":"ContainerStarted","Data":"1f545e9417c9a221a9a4415f4b519ae84de0d955201aa6a890fb777714af843f"} Jul 24 03:27:22.076638 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:22.076587 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-crlwn" podStartSLOduration=1.092144419 podStartE2EDuration="4.076573773s" totalImagesPullingTime="2.984429354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:27:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:27:22.075901446 +0000 UTC m=+539.351472797" watchObservedRunningTime="2026-07-24 03:27:22.076573773 +0000 UTC m=+539.352145122" Jul 24 03:27:23.608067 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:23.608034 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-crlwn"] Jul 24 03:27:24.071416 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:24.071337 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-crlwn" podUID="7160b880-9e04-45e9-a5d4-f47c2c6d645a" containerName="authorino" containerID="cri-o://1f545e9417c9a221a9a4415f4b519ae84de0d955201aa6a890fb777714af843f" gracePeriod=30 Jul 24 03:27:24.314952 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:24.314928 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-crlwn" Jul 24 03:27:24.404344 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:24.404314 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wkhcf\" (UniqueName: \"kubernetes.io/projected/7160b880-9e04-45e9-a5d4-f47c2c6d645a-kube-api-access-wkhcf\") pod \"7160b880-9e04-45e9-a5d4-f47c2c6d645a\" (UID: \"7160b880-9e04-45e9-a5d4-f47c2c6d645a\") " Jul 24 03:27:24.406320 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:24.406298 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7160b880-9e04-45e9-a5d4-f47c2c6d645a-kube-api-access-wkhcf" (OuterVolumeSpecName: "kube-api-access-wkhcf") pod "7160b880-9e04-45e9-a5d4-f47c2c6d645a" (UID: "7160b880-9e04-45e9-a5d4-f47c2c6d645a"). InnerVolumeSpecName "kube-api-access-wkhcf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 03:27:24.505853 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:24.505830 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkhcf\" (UniqueName: \"kubernetes.io/projected/7160b880-9e04-45e9-a5d4-f47c2c6d645a-kube-api-access-wkhcf\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:27:25.076832 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:25.076800 2579 generic.go:356] "Generic (PLEG): container finished" podID="7160b880-9e04-45e9-a5d4-f47c2c6d645a" containerID="1f545e9417c9a221a9a4415f4b519ae84de0d955201aa6a890fb777714af843f" exitCode=0 Jul 24 03:27:25.077219 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:25.076841 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-crlwn" Jul 24 03:27:25.077219 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:25.076857 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-crlwn" event={"ID":"7160b880-9e04-45e9-a5d4-f47c2c6d645a","Type":"ContainerDied","Data":"1f545e9417c9a221a9a4415f4b519ae84de0d955201aa6a890fb777714af843f"} Jul 24 03:27:25.077219 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:25.076899 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-crlwn" event={"ID":"7160b880-9e04-45e9-a5d4-f47c2c6d645a","Type":"ContainerDied","Data":"5f8cc47296a0a95d3706988d1ea1bfedb6d312f32e4358beffd158e4c3e6cf58"} Jul 24 03:27:25.077219 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:25.076918 2579 scope.go:117] "RemoveContainer" containerID="1f545e9417c9a221a9a4415f4b519ae84de0d955201aa6a890fb777714af843f" Jul 24 03:27:25.085068 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:25.085051 2579 scope.go:117] "RemoveContainer" containerID="1f545e9417c9a221a9a4415f4b519ae84de0d955201aa6a890fb777714af843f" Jul 24 03:27:25.085316 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:27:25.085297 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f545e9417c9a221a9a4415f4b519ae84de0d955201aa6a890fb777714af843f\": container with ID starting with 1f545e9417c9a221a9a4415f4b519ae84de0d955201aa6a890fb777714af843f not found: ID does not exist" containerID="1f545e9417c9a221a9a4415f4b519ae84de0d955201aa6a890fb777714af843f" Jul 24 03:27:25.085369 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:25.085324 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f545e9417c9a221a9a4415f4b519ae84de0d955201aa6a890fb777714af843f"} err="failed to get container status \"1f545e9417c9a221a9a4415f4b519ae84de0d955201aa6a890fb777714af843f\": rpc error: code = NotFound desc = could not find container \"1f545e9417c9a221a9a4415f4b519ae84de0d955201aa6a890fb777714af843f\": container with ID starting with 1f545e9417c9a221a9a4415f4b519ae84de0d955201aa6a890fb777714af843f not found: ID does not exist" Jul 24 03:27:25.095792 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:25.095769 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-crlwn"] Jul 24 03:27:25.099460 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:25.099440 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-crlwn"] Jul 24 03:27:25.520888 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:25.520814 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7160b880-9e04-45e9-a5d4-f47c2c6d645a" path="/var/lib/kubelet/pods/7160b880-9e04-45e9-a5d4-f47c2c6d645a/volumes" Jul 24 03:27:51.448507 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:51.448476 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-9cff9cb9b-k7nhm"] Jul 24 03:27:51.449005 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:51.448802 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7160b880-9e04-45e9-a5d4-f47c2c6d645a" containerName="authorino" Jul 24 03:27:51.449005 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:51.448815 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7160b880-9e04-45e9-a5d4-f47c2c6d645a" containerName="authorino" Jul 24 03:27:51.449005 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:51.448875 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7160b880-9e04-45e9-a5d4-f47c2c6d645a" containerName="authorino" Jul 24 03:27:51.450813 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:51.450797 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9cff9cb9b-k7nhm" Jul 24 03:27:51.453533 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:51.453511 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 24 03:27:51.453644 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:51.453567 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-jdbv2\"" Jul 24 03:27:51.458567 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:51.458547 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-9cff9cb9b-k7nhm"] Jul 24 03:27:51.522859 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:51.522830 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/857ab35f-3e09-401f-893d-aaf6de4996d6-tls-cert\") pod \"authorino-9cff9cb9b-k7nhm\" (UID: \"857ab35f-3e09-401f-893d-aaf6de4996d6\") " pod="kuadrant-system/authorino-9cff9cb9b-k7nhm" Jul 24 03:27:51.522982 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:51.522879 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swkrz\" (UniqueName: \"kubernetes.io/projected/857ab35f-3e09-401f-893d-aaf6de4996d6-kube-api-access-swkrz\") pod \"authorino-9cff9cb9b-k7nhm\" (UID: \"857ab35f-3e09-401f-893d-aaf6de4996d6\") " pod="kuadrant-system/authorino-9cff9cb9b-k7nhm" Jul 24 03:27:51.623725 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:51.623698 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/857ab35f-3e09-401f-893d-aaf6de4996d6-tls-cert\") pod \"authorino-9cff9cb9b-k7nhm\" (UID: \"857ab35f-3e09-401f-893d-aaf6de4996d6\") " pod="kuadrant-system/authorino-9cff9cb9b-k7nhm" Jul 24 03:27:51.623808 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:51.623747 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swkrz\" (UniqueName: \"kubernetes.io/projected/857ab35f-3e09-401f-893d-aaf6de4996d6-kube-api-access-swkrz\") pod \"authorino-9cff9cb9b-k7nhm\" (UID: \"857ab35f-3e09-401f-893d-aaf6de4996d6\") " pod="kuadrant-system/authorino-9cff9cb9b-k7nhm" Jul 24 03:27:51.626006 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:51.625980 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/857ab35f-3e09-401f-893d-aaf6de4996d6-tls-cert\") pod \"authorino-9cff9cb9b-k7nhm\" (UID: \"857ab35f-3e09-401f-893d-aaf6de4996d6\") " pod="kuadrant-system/authorino-9cff9cb9b-k7nhm" Jul 24 03:27:51.631058 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:51.631032 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swkrz\" (UniqueName: \"kubernetes.io/projected/857ab35f-3e09-401f-893d-aaf6de4996d6-kube-api-access-swkrz\") pod \"authorino-9cff9cb9b-k7nhm\" (UID: \"857ab35f-3e09-401f-893d-aaf6de4996d6\") " pod="kuadrant-system/authorino-9cff9cb9b-k7nhm" Jul 24 03:27:51.760713 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:51.760627 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9cff9cb9b-k7nhm" Jul 24 03:27:51.883489 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:51.883464 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-9cff9cb9b-k7nhm"] Jul 24 03:27:51.885589 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:27:51.885563 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod857ab35f_3e09_401f_893d_aaf6de4996d6.slice/crio-8d10cedc7d45ce20e24e002167aa2c2c812dbe4b6d0fe21dcf5a94188d388803 WatchSource:0}: Error finding container 8d10cedc7d45ce20e24e002167aa2c2c812dbe4b6d0fe21dcf5a94188d388803: Status 404 returned error can't find the container with id 8d10cedc7d45ce20e24e002167aa2c2c812dbe4b6d0fe21dcf5a94188d388803 Jul 24 03:27:52.198049 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:52.198011 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9cff9cb9b-k7nhm" event={"ID":"857ab35f-3e09-401f-893d-aaf6de4996d6","Type":"ContainerStarted","Data":"8d10cedc7d45ce20e24e002167aa2c2c812dbe4b6d0fe21dcf5a94188d388803"} Jul 24 03:27:53.203484 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:53.203447 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9cff9cb9b-k7nhm" event={"ID":"857ab35f-3e09-401f-893d-aaf6de4996d6","Type":"ContainerStarted","Data":"be31a3b56d2eebc8b12829bc9e94af73c93f8e64af7cb918473cba8f2c6e0b50"} Jul 24 03:27:53.218945 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:27:53.218892 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-9cff9cb9b-k7nhm" podStartSLOduration=1.879538829 podStartE2EDuration="2.218873288s" totalImagesPullingTime="339.334459ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:27:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:27:53.217717545 +0000 UTC m=+570.493288895" watchObservedRunningTime="2026-07-24 03:27:53.218873288 +0000 UTC m=+570.494444640" Jul 24 03:28:23.328621 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:28:23.328595 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ffrlv_6b2e05ef-119c-4450-9827-f806c437f29f/ovn-acl-logging/0.log" Jul 24 03:28:23.332891 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:28:23.332867 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ffrlv_6b2e05ef-119c-4450-9827-f806c437f29f/ovn-acl-logging/0.log" Jul 24 03:29:51.445520 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:29:51.445488 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85"] Jul 24 03:29:51.449406 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:29:51.449386 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85" Jul 24 03:29:51.452462 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:29:51.452437 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 24 03:29:51.452574 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:29:51.452459 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-kdtrl\"" Jul 24 03:29:51.452574 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:29:51.452437 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 24 03:29:51.452574 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:29:51.452475 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 24 03:29:51.464909 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:29:51.464886 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85"] Jul 24 03:29:51.523623 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:29:51.523586 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2365c615-fd5a-43df-a6a2-754cf23daba3-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85\" (UID: \"2365c615-fd5a-43df-a6a2-754cf23daba3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85" Jul 24 03:29:51.523623 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:29:51.523625 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2365c615-fd5a-43df-a6a2-754cf23daba3-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85\" (UID: \"2365c615-fd5a-43df-a6a2-754cf23daba3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85" Jul 24 03:29:51.523816 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:29:51.523683 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2365c615-fd5a-43df-a6a2-754cf23daba3-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85\" (UID: \"2365c615-fd5a-43df-a6a2-754cf23daba3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85" Jul 24 03:29:51.523816 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:29:51.523708 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2365c615-fd5a-43df-a6a2-754cf23daba3-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85\" (UID: \"2365c615-fd5a-43df-a6a2-754cf23daba3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85" Jul 24 03:29:51.523816 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:29:51.523729 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2365c615-fd5a-43df-a6a2-754cf23daba3-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85\" (UID: \"2365c615-fd5a-43df-a6a2-754cf23daba3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85" Jul 24 03:29:51.523816 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:29:51.523766 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsz4l\" (UniqueName: \"kubernetes.io/projected/2365c615-fd5a-43df-a6a2-754cf23daba3-kube-api-access-zsz4l\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85\" (UID: \"2365c615-fd5a-43df-a6a2-754cf23daba3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85" Jul 24 03:29:51.624341 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:29:51.624310 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2365c615-fd5a-43df-a6a2-754cf23daba3-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85\" (UID: \"2365c615-fd5a-43df-a6a2-754cf23daba3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85" Jul 24 03:29:51.624341 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:29:51.624343 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2365c615-fd5a-43df-a6a2-754cf23daba3-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85\" (UID: \"2365c615-fd5a-43df-a6a2-754cf23daba3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85" Jul 24 03:29:51.624341 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:29:51.624366 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2365c615-fd5a-43df-a6a2-754cf23daba3-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85\" (UID: \"2365c615-fd5a-43df-a6a2-754cf23daba3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85" Jul 24 03:29:51.624682 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:29:51.624388 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2365c615-fd5a-43df-a6a2-754cf23daba3-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85\" (UID: \"2365c615-fd5a-43df-a6a2-754cf23daba3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85" Jul 24 03:29:51.624682 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:29:51.624421 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2365c615-fd5a-43df-a6a2-754cf23daba3-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85\" (UID: \"2365c615-fd5a-43df-a6a2-754cf23daba3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85" Jul 24 03:29:51.624682 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:29:51.624445 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsz4l\" (UniqueName: \"kubernetes.io/projected/2365c615-fd5a-43df-a6a2-754cf23daba3-kube-api-access-zsz4l\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85\" (UID: \"2365c615-fd5a-43df-a6a2-754cf23daba3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85" Jul 24 03:29:51.624869 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:29:51.624825 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2365c615-fd5a-43df-a6a2-754cf23daba3-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85\" (UID: \"2365c615-fd5a-43df-a6a2-754cf23daba3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85" Jul 24 03:29:51.624935 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:29:51.624883 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2365c615-fd5a-43df-a6a2-754cf23daba3-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85\" (UID: \"2365c615-fd5a-43df-a6a2-754cf23daba3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85" Jul 24 03:29:51.625050 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:29:51.625027 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2365c615-fd5a-43df-a6a2-754cf23daba3-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85\" (UID: \"2365c615-fd5a-43df-a6a2-754cf23daba3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85" Jul 24 03:29:51.626770 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:29:51.626753 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2365c615-fd5a-43df-a6a2-754cf23daba3-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85\" (UID: \"2365c615-fd5a-43df-a6a2-754cf23daba3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85" Jul 24 03:29:51.626945 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:29:51.626925 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2365c615-fd5a-43df-a6a2-754cf23daba3-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85\" (UID: \"2365c615-fd5a-43df-a6a2-754cf23daba3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85" Jul 24 03:29:51.631780 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:29:51.631758 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsz4l\" (UniqueName: \"kubernetes.io/projected/2365c615-fd5a-43df-a6a2-754cf23daba3-kube-api-access-zsz4l\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85\" (UID: \"2365c615-fd5a-43df-a6a2-754cf23daba3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85" Jul 24 03:29:51.760805 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:29:51.760725 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85" Jul 24 03:29:51.890501 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:29:51.890477 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85"] Jul 24 03:29:51.892910 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:29:51.892882 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2365c615_fd5a_43df_a6a2_754cf23daba3.slice/crio-cf0b09c02fb74f759c3ad130058da6655ad6929dd3d90dcf7dea000d7685c62d WatchSource:0}: Error finding container cf0b09c02fb74f759c3ad130058da6655ad6929dd3d90dcf7dea000d7685c62d: Status 404 returned error can't find the container with id cf0b09c02fb74f759c3ad130058da6655ad6929dd3d90dcf7dea000d7685c62d Jul 24 03:29:51.894569 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:29:51.894552 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 03:29:52.723108 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:29:52.723076 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85" event={"ID":"2365c615-fd5a-43df-a6a2-754cf23daba3","Type":"ContainerStarted","Data":"cf0b09c02fb74f759c3ad130058da6655ad6929dd3d90dcf7dea000d7685c62d"} Jul 24 03:29:53.729420 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:29:53.729383 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85" event={"ID":"2365c615-fd5a-43df-a6a2-754cf23daba3","Type":"ContainerStarted","Data":"848cd8e1d826883284132433cf274e1ed75d3545298ce42451ff715f13987b85"} Jul 24 03:29:53.750189 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:29:53.750136 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85" podStartSLOduration=1.7636922510000002 podStartE2EDuration="2.750121719s" totalImagesPullingTime="986.429468ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:29:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:29:53.748162794 +0000 UTC m=+691.023734145" watchObservedRunningTime="2026-07-24 03:29:53.750121719 +0000 UTC m=+691.025693069" Jul 24 03:30:00.142283 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:00.142236 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747730-2bvhw"] Jul 24 03:30:00.145728 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:00.145709 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747730-2bvhw" Jul 24 03:30:00.148006 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:00.147972 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cxrc8\"" Jul 24 03:30:00.150016 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:00.149990 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747730-2bvhw"] Jul 24 03:30:00.157340 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:00.157322 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747730-2bvhw" Jul 24 03:30:00.284956 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:00.284927 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747730-2bvhw"] Jul 24 03:30:00.288585 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:30:00.288562 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13fe01ea_d114_4ef3_b524_dbd4ccc7379b.slice/crio-7e8541c7ae41fb2fa1e448007f2ddc18b97e7543b0fcb3a226feb7c3517cc9e4 WatchSource:0}: Error finding container 7e8541c7ae41fb2fa1e448007f2ddc18b97e7543b0fcb3a226feb7c3517cc9e4: Status 404 returned error can't find the container with id 7e8541c7ae41fb2fa1e448007f2ddc18b97e7543b0fcb3a226feb7c3517cc9e4 Jul 24 03:30:00.762332 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:00.762294 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747730-2bvhw" event={"ID":"13fe01ea-d114-4ef3-b524-dbd4ccc7379b","Type":"ContainerStarted","Data":"7e8541c7ae41fb2fa1e448007f2ddc18b97e7543b0fcb3a226feb7c3517cc9e4"} Jul 24 03:30:01.761629 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:01.761593 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85" Jul 24 03:30:01.762066 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:01.761685 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85" Jul 24 03:30:01.778469 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:01.778438 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85" Jul 24 03:30:01.791811 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:01.791779 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85" Jul 24 03:30:02.774704 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:02.774613 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747730-2bvhw" event={"ID":"13fe01ea-d114-4ef3-b524-dbd4ccc7379b","Type":"ContainerStarted","Data":"29940d971a83ecdd6c8e2001ee652672a9c88cf9e49c924cbe48a5f13f6cbaf9"} Jul 24 03:30:02.792567 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:02.792525 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747730-2bvhw" podStartSLOduration=0.559729799 podStartE2EDuration="2.792508059s" totalImagesPullingTime="2.23277826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:30:02.791343773 +0000 UTC m=+700.066915136" watchObservedRunningTime="2026-07-24 03:30:02.792508059 +0000 UTC m=+700.068079409" Jul 24 03:30:03.780308 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:03.780210 2579 generic.go:356] "Generic (PLEG): container finished" podID="13fe01ea-d114-4ef3-b524-dbd4ccc7379b" containerID="29940d971a83ecdd6c8e2001ee652672a9c88cf9e49c924cbe48a5f13f6cbaf9" exitCode=0 Jul 24 03:30:03.780308 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:03.780294 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747730-2bvhw" event={"ID":"13fe01ea-d114-4ef3-b524-dbd4ccc7379b","Type":"ContainerDied","Data":"29940d971a83ecdd6c8e2001ee652672a9c88cf9e49c924cbe48a5f13f6cbaf9"} Jul 24 03:30:04.913598 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:04.913571 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747730-2bvhw" Jul 24 03:30:05.258574 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:05.258543 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm"] Jul 24 03:30:05.258924 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:05.258910 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13fe01ea-d114-4ef3-b524-dbd4ccc7379b" containerName="cleanup" Jul 24 03:30:05.258981 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:05.258926 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="13fe01ea-d114-4ef3-b524-dbd4ccc7379b" containerName="cleanup" Jul 24 03:30:05.259013 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:05.258993 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="13fe01ea-d114-4ef3-b524-dbd4ccc7379b" containerName="cleanup" Jul 24 03:30:05.261977 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:05.261959 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm" Jul 24 03:30:05.263919 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:05.263901 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 24 03:30:05.271126 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:05.271104 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm"] Jul 24 03:30:05.345132 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:05.345096 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51ef15da-cbda-4004-8d20-04aa58bcaa60-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm\" (UID: \"51ef15da-cbda-4004-8d20-04aa58bcaa60\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm" Jul 24 03:30:05.345132 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:05.345133 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvqnw\" (UniqueName: \"kubernetes.io/projected/51ef15da-cbda-4004-8d20-04aa58bcaa60-kube-api-access-nvqnw\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm\" (UID: \"51ef15da-cbda-4004-8d20-04aa58bcaa60\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm" Jul 24 03:30:05.345379 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:05.345239 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51ef15da-cbda-4004-8d20-04aa58bcaa60-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm\" (UID: \"51ef15da-cbda-4004-8d20-04aa58bcaa60\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm" Jul 24 03:30:05.345379 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:05.345347 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51ef15da-cbda-4004-8d20-04aa58bcaa60-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm\" (UID: \"51ef15da-cbda-4004-8d20-04aa58bcaa60\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm" Jul 24 03:30:05.345465 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:05.345378 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51ef15da-cbda-4004-8d20-04aa58bcaa60-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm\" (UID: \"51ef15da-cbda-4004-8d20-04aa58bcaa60\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm" Jul 24 03:30:05.345465 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:05.345427 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51ef15da-cbda-4004-8d20-04aa58bcaa60-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm\" (UID: \"51ef15da-cbda-4004-8d20-04aa58bcaa60\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm" Jul 24 03:30:05.446519 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:05.446480 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51ef15da-cbda-4004-8d20-04aa58bcaa60-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm\" (UID: \"51ef15da-cbda-4004-8d20-04aa58bcaa60\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm" Jul 24 03:30:05.446519 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:05.446526 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51ef15da-cbda-4004-8d20-04aa58bcaa60-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm\" (UID: \"51ef15da-cbda-4004-8d20-04aa58bcaa60\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm" Jul 24 03:30:05.446892 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:05.446550 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51ef15da-cbda-4004-8d20-04aa58bcaa60-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm\" (UID: \"51ef15da-cbda-4004-8d20-04aa58bcaa60\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm" Jul 24 03:30:05.446892 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:05.446569 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvqnw\" (UniqueName: \"kubernetes.io/projected/51ef15da-cbda-4004-8d20-04aa58bcaa60-kube-api-access-nvqnw\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm\" (UID: \"51ef15da-cbda-4004-8d20-04aa58bcaa60\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm" Jul 24 03:30:05.446892 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:05.446618 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51ef15da-cbda-4004-8d20-04aa58bcaa60-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm\" (UID: \"51ef15da-cbda-4004-8d20-04aa58bcaa60\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm" Jul 24 03:30:05.446892 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:05.446691 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51ef15da-cbda-4004-8d20-04aa58bcaa60-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm\" (UID: \"51ef15da-cbda-4004-8d20-04aa58bcaa60\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm" Jul 24 03:30:05.447102 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:05.447047 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51ef15da-cbda-4004-8d20-04aa58bcaa60-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm\" (UID: \"51ef15da-cbda-4004-8d20-04aa58bcaa60\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm" Jul 24 03:30:05.447144 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:05.447099 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51ef15da-cbda-4004-8d20-04aa58bcaa60-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm\" (UID: \"51ef15da-cbda-4004-8d20-04aa58bcaa60\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm" Jul 24 03:30:05.447208 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:05.447191 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51ef15da-cbda-4004-8d20-04aa58bcaa60-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm\" (UID: \"51ef15da-cbda-4004-8d20-04aa58bcaa60\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm" Jul 24 03:30:05.449382 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:05.449354 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51ef15da-cbda-4004-8d20-04aa58bcaa60-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm\" (UID: \"51ef15da-cbda-4004-8d20-04aa58bcaa60\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm" Jul 24 03:30:05.449516 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:05.449498 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51ef15da-cbda-4004-8d20-04aa58bcaa60-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm\" (UID: \"51ef15da-cbda-4004-8d20-04aa58bcaa60\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm" Jul 24 03:30:05.453580 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:05.453559 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvqnw\" (UniqueName: \"kubernetes.io/projected/51ef15da-cbda-4004-8d20-04aa58bcaa60-kube-api-access-nvqnw\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm\" (UID: \"51ef15da-cbda-4004-8d20-04aa58bcaa60\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm" Jul 24 03:30:05.571792 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:05.571754 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm" Jul 24 03:30:05.704947 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:05.704923 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm"] Jul 24 03:30:05.706835 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:30:05.706812 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod51ef15da_cbda_4004_8d20_04aa58bcaa60.slice/crio-e5645a3d1afa323058fe4b5929cc9264795956d9c7e7aca1373883cacda50575 WatchSource:0}: Error finding container e5645a3d1afa323058fe4b5929cc9264795956d9c7e7aca1373883cacda50575: Status 404 returned error can't find the container with id e5645a3d1afa323058fe4b5929cc9264795956d9c7e7aca1373883cacda50575 Jul 24 03:30:05.789888 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:05.789797 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm" event={"ID":"51ef15da-cbda-4004-8d20-04aa58bcaa60","Type":"ContainerStarted","Data":"e5645a3d1afa323058fe4b5929cc9264795956d9c7e7aca1373883cacda50575"} Jul 24 03:30:05.791005 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:05.790975 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747730-2bvhw" Jul 24 03:30:05.791005 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:05.790990 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747730-2bvhw" event={"ID":"13fe01ea-d114-4ef3-b524-dbd4ccc7379b","Type":"ContainerDied","Data":"7e8541c7ae41fb2fa1e448007f2ddc18b97e7543b0fcb3a226feb7c3517cc9e4"} Jul 24 03:30:05.791158 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:05.791030 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7e8541c7ae41fb2fa1e448007f2ddc18b97e7543b0fcb3a226feb7c3517cc9e4" Jul 24 03:30:06.796854 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:06.796817 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm" event={"ID":"51ef15da-cbda-4004-8d20-04aa58bcaa60","Type":"ContainerStarted","Data":"7c6b0f78466ee7097e134de290f84e379aea45f90d8a6bc5377f204e9f22d51a"} Jul 24 03:30:06.813597 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:06.813549 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm" podStartSLOduration=1.625154459 podStartE2EDuration="1.813534179s" totalImagesPullingTime="188.37972ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:30:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:30:06.812163515 +0000 UTC m=+704.087734869" watchObservedRunningTime="2026-07-24 03:30:06.813534179 +0000 UTC m=+704.089105530" Jul 24 03:30:15.572829 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:15.572743 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm" Jul 24 03:30:15.572829 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:15.572797 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm" Jul 24 03:30:15.584752 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:15.584724 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm" Jul 24 03:30:15.852888 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:15.852858 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm" Jul 24 03:30:58.705242 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:58.705202 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-9cff9cb9b-k7nhm"] Jul 24 03:30:58.705751 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:58.705439 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-9cff9cb9b-k7nhm" podUID="857ab35f-3e09-401f-893d-aaf6de4996d6" containerName="authorino" containerID="cri-o://be31a3b56d2eebc8b12829bc9e94af73c93f8e64af7cb918473cba8f2c6e0b50" gracePeriod=30 Jul 24 03:30:58.959137 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:58.959085 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9cff9cb9b-k7nhm" Jul 24 03:30:59.003051 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:59.003014 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/857ab35f-3e09-401f-893d-aaf6de4996d6-tls-cert\") pod \"857ab35f-3e09-401f-893d-aaf6de4996d6\" (UID: \"857ab35f-3e09-401f-893d-aaf6de4996d6\") " Jul 24 03:30:59.003228 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:59.003077 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-swkrz\" (UniqueName: \"kubernetes.io/projected/857ab35f-3e09-401f-893d-aaf6de4996d6-kube-api-access-swkrz\") pod \"857ab35f-3e09-401f-893d-aaf6de4996d6\" (UID: \"857ab35f-3e09-401f-893d-aaf6de4996d6\") " Jul 24 03:30:59.005016 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:59.004989 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/857ab35f-3e09-401f-893d-aaf6de4996d6-kube-api-access-swkrz" (OuterVolumeSpecName: "kube-api-access-swkrz") pod "857ab35f-3e09-401f-893d-aaf6de4996d6" (UID: "857ab35f-3e09-401f-893d-aaf6de4996d6"). InnerVolumeSpecName "kube-api-access-swkrz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 03:30:59.012705 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:59.012655 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/857ab35f-3e09-401f-893d-aaf6de4996d6-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "857ab35f-3e09-401f-893d-aaf6de4996d6" (UID: "857ab35f-3e09-401f-893d-aaf6de4996d6"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 03:30:59.039376 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:59.039340 2579 generic.go:356] "Generic (PLEG): container finished" podID="857ab35f-3e09-401f-893d-aaf6de4996d6" containerID="be31a3b56d2eebc8b12829bc9e94af73c93f8e64af7cb918473cba8f2c6e0b50" exitCode=0 Jul 24 03:30:59.039496 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:59.039387 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9cff9cb9b-k7nhm" Jul 24 03:30:59.039496 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:59.039423 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9cff9cb9b-k7nhm" event={"ID":"857ab35f-3e09-401f-893d-aaf6de4996d6","Type":"ContainerDied","Data":"be31a3b56d2eebc8b12829bc9e94af73c93f8e64af7cb918473cba8f2c6e0b50"} Jul 24 03:30:59.039496 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:59.039471 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9cff9cb9b-k7nhm" event={"ID":"857ab35f-3e09-401f-893d-aaf6de4996d6","Type":"ContainerDied","Data":"8d10cedc7d45ce20e24e002167aa2c2c812dbe4b6d0fe21dcf5a94188d388803"} Jul 24 03:30:59.039496 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:59.039490 2579 scope.go:117] "RemoveContainer" containerID="be31a3b56d2eebc8b12829bc9e94af73c93f8e64af7cb918473cba8f2c6e0b50" Jul 24 03:30:59.050760 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:59.050719 2579 scope.go:117] "RemoveContainer" containerID="be31a3b56d2eebc8b12829bc9e94af73c93f8e64af7cb918473cba8f2c6e0b50" Jul 24 03:30:59.051345 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:30:59.051326 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be31a3b56d2eebc8b12829bc9e94af73c93f8e64af7cb918473cba8f2c6e0b50\": container with ID starting with be31a3b56d2eebc8b12829bc9e94af73c93f8e64af7cb918473cba8f2c6e0b50 not found: ID does not exist" containerID="be31a3b56d2eebc8b12829bc9e94af73c93f8e64af7cb918473cba8f2c6e0b50" Jul 24 03:30:59.051416 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:59.051351 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be31a3b56d2eebc8b12829bc9e94af73c93f8e64af7cb918473cba8f2c6e0b50"} err="failed to get container status \"be31a3b56d2eebc8b12829bc9e94af73c93f8e64af7cb918473cba8f2c6e0b50\": rpc error: code = NotFound desc = could not find container \"be31a3b56d2eebc8b12829bc9e94af73c93f8e64af7cb918473cba8f2c6e0b50\": container with ID starting with be31a3b56d2eebc8b12829bc9e94af73c93f8e64af7cb918473cba8f2c6e0b50 not found: ID does not exist" Jul 24 03:30:59.060252 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:59.060224 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-9cff9cb9b-k7nhm"] Jul 24 03:30:59.062768 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:59.062749 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-9cff9cb9b-k7nhm"] Jul 24 03:30:59.104259 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:59.104226 2579 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/857ab35f-3e09-401f-893d-aaf6de4996d6-tls-cert\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:30:59.104259 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:59.104253 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-swkrz\" (UniqueName: \"kubernetes.io/projected/857ab35f-3e09-401f-893d-aaf6de4996d6-kube-api-access-swkrz\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:30:59.521932 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:30:59.521903 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="857ab35f-3e09-401f-893d-aaf6de4996d6" path="/var/lib/kubelet/pods/857ab35f-3e09-401f-893d-aaf6de4996d6/volumes" Jul 24 03:33:23.358771 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:33:23.358741 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ffrlv_6b2e05ef-119c-4450-9827-f806c437f29f/ovn-acl-logging/0.log" Jul 24 03:33:23.361911 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:33:23.361889 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ffrlv_6b2e05ef-119c-4450-9827-f806c437f29f/ovn-acl-logging/0.log" Jul 24 03:37:55.861834 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:37:55.861802 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gmb7f"] Jul 24 03:37:55.862410 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:37:55.862051 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gmb7f" podUID="a341ebec-2e2f-4178-86ee-17ecf3f5613f" containerName="manager" containerID="cri-o://52dfabd4ff9435dd76f50cea1cd32b94e21041ce5d486b5d1b7df7a8fac80501" gracePeriod=10 Jul 24 03:37:58.311428 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:37:58.311407 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gmb7f" Jul 24 03:37:58.364918 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:37:58.364892 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-58wzf\" (UniqueName: \"kubernetes.io/projected/a341ebec-2e2f-4178-86ee-17ecf3f5613f-kube-api-access-58wzf\") pod \"a341ebec-2e2f-4178-86ee-17ecf3f5613f\" (UID: \"a341ebec-2e2f-4178-86ee-17ecf3f5613f\") " Jul 24 03:37:58.365039 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:37:58.364951 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a341ebec-2e2f-4178-86ee-17ecf3f5613f-extensions-socket-volume\") pod \"a341ebec-2e2f-4178-86ee-17ecf3f5613f\" (UID: \"a341ebec-2e2f-4178-86ee-17ecf3f5613f\") " Jul 24 03:37:58.365386 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:37:58.365355 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a341ebec-2e2f-4178-86ee-17ecf3f5613f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a341ebec-2e2f-4178-86ee-17ecf3f5613f" (UID: "a341ebec-2e2f-4178-86ee-17ecf3f5613f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 03:37:58.366942 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:37:58.366922 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a341ebec-2e2f-4178-86ee-17ecf3f5613f-kube-api-access-58wzf" (OuterVolumeSpecName: "kube-api-access-58wzf") pod "a341ebec-2e2f-4178-86ee-17ecf3f5613f" (UID: "a341ebec-2e2f-4178-86ee-17ecf3f5613f"). InnerVolumeSpecName "kube-api-access-58wzf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 03:37:58.466278 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:37:58.466208 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58wzf\" (UniqueName: \"kubernetes.io/projected/a341ebec-2e2f-4178-86ee-17ecf3f5613f-kube-api-access-58wzf\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:37:58.466278 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:37:58.466233 2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a341ebec-2e2f-4178-86ee-17ecf3f5613f-extensions-socket-volume\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:37:58.918584 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:37:58.918543 2579 generic.go:356] "Generic (PLEG): container finished" podID="a341ebec-2e2f-4178-86ee-17ecf3f5613f" containerID="52dfabd4ff9435dd76f50cea1cd32b94e21041ce5d486b5d1b7df7a8fac80501" exitCode=0 Jul 24 03:37:58.918776 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:37:58.918590 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gmb7f" Jul 24 03:37:58.918776 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:37:58.918628 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gmb7f" event={"ID":"a341ebec-2e2f-4178-86ee-17ecf3f5613f","Type":"ContainerDied","Data":"52dfabd4ff9435dd76f50cea1cd32b94e21041ce5d486b5d1b7df7a8fac80501"} Jul 24 03:37:58.918776 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:37:58.918701 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gmb7f" event={"ID":"a341ebec-2e2f-4178-86ee-17ecf3f5613f","Type":"ContainerDied","Data":"c60a4796e9968ff3fdba942f1611bdb18494f1c6efc652b16e0bcc43371a66be"} Jul 24 03:37:58.918776 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:37:58.918725 2579 scope.go:117] "RemoveContainer" containerID="52dfabd4ff9435dd76f50cea1cd32b94e21041ce5d486b5d1b7df7a8fac80501" Jul 24 03:37:58.928440 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:37:58.928418 2579 scope.go:117] "RemoveContainer" containerID="52dfabd4ff9435dd76f50cea1cd32b94e21041ce5d486b5d1b7df7a8fac80501" Jul 24 03:37:58.928718 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:37:58.928699 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52dfabd4ff9435dd76f50cea1cd32b94e21041ce5d486b5d1b7df7a8fac80501\": container with ID starting with 52dfabd4ff9435dd76f50cea1cd32b94e21041ce5d486b5d1b7df7a8fac80501 not found: ID does not exist" containerID="52dfabd4ff9435dd76f50cea1cd32b94e21041ce5d486b5d1b7df7a8fac80501" Jul 24 03:37:58.928763 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:37:58.928728 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52dfabd4ff9435dd76f50cea1cd32b94e21041ce5d486b5d1b7df7a8fac80501"} err="failed to get container status \"52dfabd4ff9435dd76f50cea1cd32b94e21041ce5d486b5d1b7df7a8fac80501\": rpc error: code = NotFound desc = could not find container \"52dfabd4ff9435dd76f50cea1cd32b94e21041ce5d486b5d1b7df7a8fac80501\": container with ID starting with 52dfabd4ff9435dd76f50cea1cd32b94e21041ce5d486b5d1b7df7a8fac80501 not found: ID does not exist" Jul 24 03:37:58.939591 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:37:58.939562 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gmb7f"] Jul 24 03:37:58.941540 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:37:58.941520 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gmb7f"] Jul 24 03:37:59.521064 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:37:59.521024 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a341ebec-2e2f-4178-86ee-17ecf3f5613f" path="/var/lib/kubelet/pods/a341ebec-2e2f-4178-86ee-17ecf3f5613f/volumes" Jul 24 03:38:23.387697 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:38:23.387648 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ffrlv_6b2e05ef-119c-4450-9827-f806c437f29f/ovn-acl-logging/0.log" Jul 24 03:38:23.388138 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:38:23.388125 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ffrlv_6b2e05ef-119c-4450-9827-f806c437f29f/ovn-acl-logging/0.log" Jul 24 03:38:57.018203 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:38:57.018167 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x69rz"] Jul 24 03:38:57.018693 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:38:57.018507 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="857ab35f-3e09-401f-893d-aaf6de4996d6" containerName="authorino" Jul 24 03:38:57.018693 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:38:57.018519 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="857ab35f-3e09-401f-893d-aaf6de4996d6" containerName="authorino" Jul 24 03:38:57.018693 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:38:57.018538 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a341ebec-2e2f-4178-86ee-17ecf3f5613f" containerName="manager" Jul 24 03:38:57.018693 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:38:57.018544 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a341ebec-2e2f-4178-86ee-17ecf3f5613f" containerName="manager" Jul 24 03:38:57.018693 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:38:57.018595 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="857ab35f-3e09-401f-893d-aaf6de4996d6" containerName="authorino" Jul 24 03:38:57.018693 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:38:57.018607 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a341ebec-2e2f-4178-86ee-17ecf3f5613f" containerName="manager" Jul 24 03:38:57.021413 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:38:57.021398 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x69rz" Jul 24 03:38:57.024314 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:38:57.024294 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-769mn\"" Jul 24 03:38:57.044912 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:38:57.044887 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x69rz"] Jul 24 03:38:57.052430 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:38:57.052404 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qpfl\" (UniqueName: \"kubernetes.io/projected/e016a7ba-4041-448e-a651-65655b698f6d-kube-api-access-6qpfl\") pod \"kuadrant-operator-controller-manager-86878c769f-x69rz\" (UID: \"e016a7ba-4041-448e-a651-65655b698f6d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x69rz" Jul 24 03:38:57.052555 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:38:57.052535 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e016a7ba-4041-448e-a651-65655b698f6d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-x69rz\" (UID: \"e016a7ba-4041-448e-a651-65655b698f6d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x69rz" Jul 24 03:38:57.153271 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:38:57.153241 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qpfl\" (UniqueName: \"kubernetes.io/projected/e016a7ba-4041-448e-a651-65655b698f6d-kube-api-access-6qpfl\") pod \"kuadrant-operator-controller-manager-86878c769f-x69rz\" (UID: \"e016a7ba-4041-448e-a651-65655b698f6d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x69rz" Jul 24 03:38:57.153381 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:38:57.153318 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e016a7ba-4041-448e-a651-65655b698f6d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-x69rz\" (UID: \"e016a7ba-4041-448e-a651-65655b698f6d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x69rz" Jul 24 03:38:57.153634 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:38:57.153619 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e016a7ba-4041-448e-a651-65655b698f6d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-x69rz\" (UID: \"e016a7ba-4041-448e-a651-65655b698f6d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x69rz" Jul 24 03:38:57.163074 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:38:57.163044 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qpfl\" (UniqueName: \"kubernetes.io/projected/e016a7ba-4041-448e-a651-65655b698f6d-kube-api-access-6qpfl\") pod \"kuadrant-operator-controller-manager-86878c769f-x69rz\" (UID: \"e016a7ba-4041-448e-a651-65655b698f6d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x69rz" Jul 24 03:38:57.331943 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:38:57.331919 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x69rz" Jul 24 03:38:57.466902 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:38:57.466876 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x69rz"] Jul 24 03:38:57.468516 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:38:57.468493 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode016a7ba_4041_448e_a651_65655b698f6d.slice/crio-1e49bb887e0d4fb206dcbe044b86d889555ae9ea60bf35c5331443a40e11fec7 WatchSource:0}: Error finding container 1e49bb887e0d4fb206dcbe044b86d889555ae9ea60bf35c5331443a40e11fec7: Status 404 returned error can't find the container with id 1e49bb887e0d4fb206dcbe044b86d889555ae9ea60bf35c5331443a40e11fec7 Jul 24 03:38:58.180836 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:38:58.180800 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x69rz" event={"ID":"e016a7ba-4041-448e-a651-65655b698f6d","Type":"ContainerStarted","Data":"02454e0365d89d36b19def5d82a002a11fcb30980ce1a94dc6d99933eb2bc0b1"} Jul 24 03:38:58.181190 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:38:58.180853 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x69rz" event={"ID":"e016a7ba-4041-448e-a651-65655b698f6d","Type":"ContainerStarted","Data":"1e49bb887e0d4fb206dcbe044b86d889555ae9ea60bf35c5331443a40e11fec7"} Jul 24 03:38:58.181190 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:38:58.180900 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x69rz" Jul 24 03:38:58.200822 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:38:58.200770 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x69rz" podStartSLOduration=2.200753829 podStartE2EDuration="2.200753829s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:38:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:38:58.199478774 +0000 UTC m=+1235.475050124" watchObservedRunningTime="2026-07-24 03:38:58.200753829 +0000 UTC m=+1235.476325179" Jul 24 03:39:09.187781 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:39:09.187746 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x69rz" Jul 24 03:41:35.279273 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:41:35.279234 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc"] Jul 24 03:41:35.282849 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:41:35.282831 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc" Jul 24 03:41:35.285416 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:41:35.285390 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-07de6cda-kserve-self-signed-certs\"" Jul 24 03:41:35.305409 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:41:35.305391 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc"] Jul 24 03:41:35.326470 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:41:35.326445 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/409f3211-0e59-4a6a-b699-60b32a513163-tmp-dir\") pod \"e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc\" (UID: \"409f3211-0e59-4a6a-b699-60b32a513163\") " pod="llm/e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc" Jul 24 03:41:35.326578 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:41:35.326485 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/409f3211-0e59-4a6a-b699-60b32a513163-home\") pod \"e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc\" (UID: \"409f3211-0e59-4a6a-b699-60b32a513163\") " pod="llm/e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc" Jul 24 03:41:35.326578 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:41:35.326503 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/409f3211-0e59-4a6a-b699-60b32a513163-dshm\") pod \"e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc\" (UID: \"409f3211-0e59-4a6a-b699-60b32a513163\") " pod="llm/e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc" Jul 24 03:41:35.326578 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:41:35.326522 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/409f3211-0e59-4a6a-b699-60b32a513163-tls-certs\") pod \"e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc\" (UID: \"409f3211-0e59-4a6a-b699-60b32a513163\") " pod="llm/e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc" Jul 24 03:41:35.326713 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:41:35.326583 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z986b\" (UniqueName: \"kubernetes.io/projected/409f3211-0e59-4a6a-b699-60b32a513163-kube-api-access-z986b\") pod \"e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc\" (UID: \"409f3211-0e59-4a6a-b699-60b32a513163\") " pod="llm/e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc" Jul 24 03:41:35.326713 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:41:35.326602 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/409f3211-0e59-4a6a-b699-60b32a513163-model-cache\") pod \"e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc\" (UID: \"409f3211-0e59-4a6a-b699-60b32a513163\") " pod="llm/e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc" Jul 24 03:41:35.427375 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:41:35.427352 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/409f3211-0e59-4a6a-b699-60b32a513163-dshm\") pod \"e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc\" (UID: \"409f3211-0e59-4a6a-b699-60b32a513163\") " pod="llm/e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc" Jul 24 03:41:35.427495 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:41:35.427380 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/409f3211-0e59-4a6a-b699-60b32a513163-tls-certs\") pod \"e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc\" (UID: \"409f3211-0e59-4a6a-b699-60b32a513163\") " pod="llm/e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc" Jul 24 03:41:35.427495 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:41:35.427404 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z986b\" (UniqueName: \"kubernetes.io/projected/409f3211-0e59-4a6a-b699-60b32a513163-kube-api-access-z986b\") pod \"e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc\" (UID: \"409f3211-0e59-4a6a-b699-60b32a513163\") " pod="llm/e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc" Jul 24 03:41:35.427495 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:41:35.427425 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/409f3211-0e59-4a6a-b699-60b32a513163-model-cache\") pod \"e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc\" (UID: \"409f3211-0e59-4a6a-b699-60b32a513163\") " pod="llm/e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc" Jul 24 03:41:35.427653 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:41:35.427509 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/409f3211-0e59-4a6a-b699-60b32a513163-tmp-dir\") pod \"e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc\" (UID: \"409f3211-0e59-4a6a-b699-60b32a513163\") " pod="llm/e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc" Jul 24 03:41:35.427653 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:41:35.427545 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/409f3211-0e59-4a6a-b699-60b32a513163-home\") pod \"e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc\" (UID: \"409f3211-0e59-4a6a-b699-60b32a513163\") " pod="llm/e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc" Jul 24 03:41:35.427921 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:41:35.427888 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/409f3211-0e59-4a6a-b699-60b32a513163-model-cache\") pod \"e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc\" (UID: \"409f3211-0e59-4a6a-b699-60b32a513163\") " pod="llm/e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc" Jul 24 03:41:35.428036 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:41:35.427932 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/409f3211-0e59-4a6a-b699-60b32a513163-home\") pod \"e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc\" (UID: \"409f3211-0e59-4a6a-b699-60b32a513163\") " pod="llm/e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc" Jul 24 03:41:35.428036 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:41:35.427954 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/409f3211-0e59-4a6a-b699-60b32a513163-tmp-dir\") pod \"e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc\" (UID: \"409f3211-0e59-4a6a-b699-60b32a513163\") " pod="llm/e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc" Jul 24 03:41:35.429657 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:41:35.429635 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/409f3211-0e59-4a6a-b699-60b32a513163-dshm\") pod \"e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc\" (UID: \"409f3211-0e59-4a6a-b699-60b32a513163\") " pod="llm/e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc" Jul 24 03:41:35.429858 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:41:35.429842 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/409f3211-0e59-4a6a-b699-60b32a513163-tls-certs\") pod \"e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc\" (UID: \"409f3211-0e59-4a6a-b699-60b32a513163\") " pod="llm/e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc" Jul 24 03:41:35.436094 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:41:35.436073 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z986b\" (UniqueName: \"kubernetes.io/projected/409f3211-0e59-4a6a-b699-60b32a513163-kube-api-access-z986b\") pod \"e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc\" (UID: \"409f3211-0e59-4a6a-b699-60b32a513163\") " pod="llm/e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc" Jul 24 03:41:35.591817 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:41:35.591786 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc" Jul 24 03:41:35.718234 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:41:35.718205 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc"] Jul 24 03:41:35.720174 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:41:35.720146 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod409f3211_0e59_4a6a_b699_60b32a513163.slice/crio-7cefaef2e31ce13d92fdb24206dd63872305302493d8ddafe9c68d7fee5c71da WatchSource:0}: Error finding container 7cefaef2e31ce13d92fdb24206dd63872305302493d8ddafe9c68d7fee5c71da: Status 404 returned error can't find the container with id 7cefaef2e31ce13d92fdb24206dd63872305302493d8ddafe9c68d7fee5c71da Jul 24 03:41:35.882352 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:41:35.882276 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc" event={"ID":"409f3211-0e59-4a6a-b699-60b32a513163","Type":"ContainerStarted","Data":"dc1aab3b832794d176301cd8a88f0c52f7f013bd6919f32fa141d57d2e1ba88f"} Jul 24 03:41:35.882352 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:41:35.882315 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc" event={"ID":"409f3211-0e59-4a6a-b699-60b32a513163","Type":"ContainerStarted","Data":"7cefaef2e31ce13d92fdb24206dd63872305302493d8ddafe9c68d7fee5c71da"} Jul 24 03:41:35.898140 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:41:35.898098 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc" podStartSLOduration=0.898075799 podStartE2EDuration="898.075799ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:41:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:41:35.896852302 +0000 UTC m=+1393.172423650" watchObservedRunningTime="2026-07-24 03:41:35.898075799 +0000 UTC m=+1393.173647149" Jul 24 03:41:45.592580 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:41:45.592551 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc" Jul 24 03:41:45.593000 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:41:45.592786 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc" Jul 24 03:41:45.604807 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:41:45.604790 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc" Jul 24 03:41:45.941871 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:41:45.941794 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc" Jul 24 03:42:24.575645 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:42:24.575564 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc"] Jul 24 03:42:39.594177 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:42:39.594123 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc" podUID="409f3211-0e59-4a6a-b699-60b32a513163" containerName="main" containerID="cri-o://dc1aab3b832794d176301cd8a88f0c52f7f013bd6919f32fa141d57d2e1ba88f" gracePeriod=45 Jul 24 03:42:39.847482 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:42:39.847425 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc" Jul 24 03:42:39.988322 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:42:39.988296 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/409f3211-0e59-4a6a-b699-60b32a513163-tmp-dir\") pod \"409f3211-0e59-4a6a-b699-60b32a513163\" (UID: \"409f3211-0e59-4a6a-b699-60b32a513163\") " Jul 24 03:42:39.988465 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:42:39.988334 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/409f3211-0e59-4a6a-b699-60b32a513163-tls-certs\") pod \"409f3211-0e59-4a6a-b699-60b32a513163\" (UID: \"409f3211-0e59-4a6a-b699-60b32a513163\") " Jul 24 03:42:39.988465 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:42:39.988372 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/409f3211-0e59-4a6a-b699-60b32a513163-model-cache\") pod \"409f3211-0e59-4a6a-b699-60b32a513163\" (UID: \"409f3211-0e59-4a6a-b699-60b32a513163\") " Jul 24 03:42:39.988465 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:42:39.988399 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z986b\" (UniqueName: \"kubernetes.io/projected/409f3211-0e59-4a6a-b699-60b32a513163-kube-api-access-z986b\") pod \"409f3211-0e59-4a6a-b699-60b32a513163\" (UID: \"409f3211-0e59-4a6a-b699-60b32a513163\") " Jul 24 03:42:39.988465 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:42:39.988418 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/409f3211-0e59-4a6a-b699-60b32a513163-home\") pod \"409f3211-0e59-4a6a-b699-60b32a513163\" (UID: \"409f3211-0e59-4a6a-b699-60b32a513163\") " Jul 24 03:42:39.988693 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:42:39.988542 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/409f3211-0e59-4a6a-b699-60b32a513163-dshm\") pod \"409f3211-0e59-4a6a-b699-60b32a513163\" (UID: \"409f3211-0e59-4a6a-b699-60b32a513163\") " Jul 24 03:42:39.988693 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:42:39.988623 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/409f3211-0e59-4a6a-b699-60b32a513163-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "409f3211-0e59-4a6a-b699-60b32a513163" (UID: "409f3211-0e59-4a6a-b699-60b32a513163"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 03:42:39.988795 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:42:39.988655 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/409f3211-0e59-4a6a-b699-60b32a513163-model-cache" (OuterVolumeSpecName: "model-cache") pod "409f3211-0e59-4a6a-b699-60b32a513163" (UID: "409f3211-0e59-4a6a-b699-60b32a513163"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 03:42:39.988795 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:42:39.988717 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/409f3211-0e59-4a6a-b699-60b32a513163-home" (OuterVolumeSpecName: "home") pod "409f3211-0e59-4a6a-b699-60b32a513163" (UID: "409f3211-0e59-4a6a-b699-60b32a513163"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 03:42:39.988926 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:42:39.988908 2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/409f3211-0e59-4a6a-b699-60b32a513163-model-cache\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:42:39.988984 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:42:39.988931 2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/409f3211-0e59-4a6a-b699-60b32a513163-home\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:42:39.988984 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:42:39.988946 2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/409f3211-0e59-4a6a-b699-60b32a513163-tmp-dir\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:42:39.990609 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:42:39.990575 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/409f3211-0e59-4a6a-b699-60b32a513163-kube-api-access-z986b" (OuterVolumeSpecName: "kube-api-access-z986b") pod "409f3211-0e59-4a6a-b699-60b32a513163" (UID: "409f3211-0e59-4a6a-b699-60b32a513163"). InnerVolumeSpecName "kube-api-access-z986b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 03:42:39.990609 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:42:39.990605 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/409f3211-0e59-4a6a-b699-60b32a513163-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "409f3211-0e59-4a6a-b699-60b32a513163" (UID: "409f3211-0e59-4a6a-b699-60b32a513163"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 03:42:39.990783 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:42:39.990648 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/409f3211-0e59-4a6a-b699-60b32a513163-dshm" (OuterVolumeSpecName: "dshm") pod "409f3211-0e59-4a6a-b699-60b32a513163" (UID: "409f3211-0e59-4a6a-b699-60b32a513163"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 03:42:40.090205 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:42:40.090175 2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/409f3211-0e59-4a6a-b699-60b32a513163-dshm\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:42:40.090205 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:42:40.090201 2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/409f3211-0e59-4a6a-b699-60b32a513163-tls-certs\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:42:40.090361 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:42:40.090213 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z986b\" (UniqueName: \"kubernetes.io/projected/409f3211-0e59-4a6a-b699-60b32a513163-kube-api-access-z986b\") on node \"ip-10-0-139-134.ec2.internal\" DevicePath \"\"" Jul 24 03:42:40.177436 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:42:40.177363 2579 generic.go:356] "Generic (PLEG): container finished" podID="409f3211-0e59-4a6a-b699-60b32a513163" containerID="dc1aab3b832794d176301cd8a88f0c52f7f013bd6919f32fa141d57d2e1ba88f" exitCode=0 Jul 24 03:42:40.177436 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:42:40.177408 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc" Jul 24 03:42:40.177578 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:42:40.177439 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc" event={"ID":"409f3211-0e59-4a6a-b699-60b32a513163","Type":"ContainerDied","Data":"dc1aab3b832794d176301cd8a88f0c52f7f013bd6919f32fa141d57d2e1ba88f"} Jul 24 03:42:40.177578 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:42:40.177476 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc" event={"ID":"409f3211-0e59-4a6a-b699-60b32a513163","Type":"ContainerDied","Data":"7cefaef2e31ce13d92fdb24206dd63872305302493d8ddafe9c68d7fee5c71da"} Jul 24 03:42:40.177578 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:42:40.177498 2579 scope.go:117] "RemoveContainer" containerID="dc1aab3b832794d176301cd8a88f0c52f7f013bd6919f32fa141d57d2e1ba88f" Jul 24 03:42:40.186461 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:42:40.186444 2579 scope.go:117] "RemoveContainer" containerID="dc1aab3b832794d176301cd8a88f0c52f7f013bd6919f32fa141d57d2e1ba88f" Jul 24 03:42:40.186711 ip-10-0-139-134 kubenswrapper[2579]: E0724 03:42:40.186692 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc1aab3b832794d176301cd8a88f0c52f7f013bd6919f32fa141d57d2e1ba88f\": container with ID starting with dc1aab3b832794d176301cd8a88f0c52f7f013bd6919f32fa141d57d2e1ba88f not found: ID does not exist" containerID="dc1aab3b832794d176301cd8a88f0c52f7f013bd6919f32fa141d57d2e1ba88f" Jul 24 03:42:40.186761 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:42:40.186720 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc1aab3b832794d176301cd8a88f0c52f7f013bd6919f32fa141d57d2e1ba88f"} err="failed to get container status \"dc1aab3b832794d176301cd8a88f0c52f7f013bd6919f32fa141d57d2e1ba88f\": rpc error: code = NotFound desc = could not find container \"dc1aab3b832794d176301cd8a88f0c52f7f013bd6919f32fa141d57d2e1ba88f\": container with ID starting with dc1aab3b832794d176301cd8a88f0c52f7f013bd6919f32fa141d57d2e1ba88f not found: ID does not exist" Jul 24 03:42:40.197171 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:42:40.197150 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc"] Jul 24 03:42:40.199853 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:42:40.199834 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-07de6cda-kserve-79f9b8954c-tnhkc"] Jul 24 03:42:41.521681 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:42:41.521639 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="409f3211-0e59-4a6a-b699-60b32a513163" path="/var/lib/kubelet/pods/409f3211-0e59-4a6a-b699-60b32a513163/volumes" Jul 24 03:43:23.415134 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:43:23.415103 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ffrlv_6b2e05ef-119c-4450-9827-f806c437f29f/ovn-acl-logging/0.log" Jul 24 03:43:23.416681 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:43:23.416651 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ffrlv_6b2e05ef-119c-4450-9827-f806c437f29f/ovn-acl-logging/0.log" Jul 24 03:45:00.139551 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:45:00.139515 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747745-ffgjc"] Jul 24 03:45:00.140051 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:45:00.139858 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="409f3211-0e59-4a6a-b699-60b32a513163" containerName="main" Jul 24 03:45:00.140051 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:45:00.139869 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="409f3211-0e59-4a6a-b699-60b32a513163" containerName="main" Jul 24 03:45:00.140051 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:45:00.139930 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="409f3211-0e59-4a6a-b699-60b32a513163" containerName="main" Jul 24 03:45:00.142933 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:45:00.142916 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747745-ffgjc" Jul 24 03:45:00.144973 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:45:00.144954 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cxrc8\"" Jul 24 03:45:00.152251 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:45:00.152232 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747745-ffgjc" Jul 24 03:45:00.158776 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:45:00.158755 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747745-ffgjc"] Jul 24 03:45:00.300172 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:45:00.300150 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747745-ffgjc"] Jul 24 03:45:00.302505 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:45:00.302472 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28a198b9_a479_4154_8ed1_60d9c3ffa0fc.slice/crio-cad5bde9a65ada17665c98de6bccb38a9f48f63540db807c9d2c2f3718ebf0f0 WatchSource:0}: Error finding container cad5bde9a65ada17665c98de6bccb38a9f48f63540db807c9d2c2f3718ebf0f0: Status 404 returned error can't find the container with id cad5bde9a65ada17665c98de6bccb38a9f48f63540db807c9d2c2f3718ebf0f0 Jul 24 03:45:00.805398 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:45:00.805309 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747745-ffgjc" event={"ID":"28a198b9-a479-4154-8ed1-60d9c3ffa0fc","Type":"ContainerStarted","Data":"8f708554ef265b03eb6037de332158d8f42442609df6bfbc9153bbb0fc7e8097"} Jul 24 03:45:00.805398 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:45:00.805383 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747745-ffgjc" event={"ID":"28a198b9-a479-4154-8ed1-60d9c3ffa0fc","Type":"ContainerStarted","Data":"cad5bde9a65ada17665c98de6bccb38a9f48f63540db807c9d2c2f3718ebf0f0"} Jul 24 03:45:00.832152 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:45:00.832094 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747745-ffgjc" podStartSLOduration=0.83207589 podStartE2EDuration="832.07589ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:45:00.82954485 +0000 UTC m=+1598.105116201" watchObservedRunningTime="2026-07-24 03:45:00.83207589 +0000 UTC m=+1598.107647241" Jul 24 03:45:01.810602 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:45:01.810566 2579 generic.go:356] "Generic (PLEG): container finished" podID="28a198b9-a479-4154-8ed1-60d9c3ffa0fc" containerID="8f708554ef265b03eb6037de332158d8f42442609df6bfbc9153bbb0fc7e8097" exitCode=0 Jul 24 03:45:01.811007 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:45:01.810625 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747745-ffgjc" event={"ID":"28a198b9-a479-4154-8ed1-60d9c3ffa0fc","Type":"ContainerDied","Data":"8f708554ef265b03eb6037de332158d8f42442609df6bfbc9153bbb0fc7e8097"} Jul 24 03:45:02.944600 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:45:02.944574 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747745-ffgjc" Jul 24 03:45:03.820186 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:45:03.820100 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747745-ffgjc" Jul 24 03:45:03.820186 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:45:03.820116 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747745-ffgjc" event={"ID":"28a198b9-a479-4154-8ed1-60d9c3ffa0fc","Type":"ContainerDied","Data":"cad5bde9a65ada17665c98de6bccb38a9f48f63540db807c9d2c2f3718ebf0f0"} Jul 24 03:45:03.820186 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:45:03.820165 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cad5bde9a65ada17665c98de6bccb38a9f48f63540db807c9d2c2f3718ebf0f0" Jul 24 03:45:39.321398 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:45:39.321316 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c75c644cf-j7hn4_b56fa8eb-8fb8-4dea-a80d-5e9dee75941e/copy-manifests/0.log" Jul 24 03:45:39.329656 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:45:39.329594 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c75c644cf-j7hn4_b56fa8eb-8fb8-4dea-a80d-5e9dee75941e/manager/0.log" Jul 24 03:45:40.023315 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:45:40.023282 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29747730-2bvhw_13fe01ea-d114-4ef3-b524-dbd4ccc7379b/cleanup/0.log" Jul 24 03:45:40.131041 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:45:40.131012 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29747745-ffgjc_28a198b9-a479-4154-8ed1-60d9c3ffa0fc/cleanup/0.log" Jul 24 03:45:41.896241 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:45:41.896212 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-vkx42_9c4703c1-46e3-4cfc-a3d2-9d9710fae811/kuadrant-console-plugin/0.log" Jul 24 03:45:42.148477 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:45:42.148390 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-x69rz_e016a7ba-4041-448e-a651-65655b698f6d/manager/0.log" Jul 24 03:45:42.989536 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:45:42.989504 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7ccbc5c95c-n9qf9_a84843d1-19b7-468e-ac6f-f925ed893a36/kube-auth-proxy/0.log" Jul 24 03:45:44.285112 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:45:44.285080 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp7g85_2365c615-fd5a-43df-a6a2-754cf23daba3/main/0.log" Jul 24 03:45:44.523469 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:45:44.523440 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-p6pzm_51ef15da-cbda-4004-8d20-04aa58bcaa60/main/0.log" Jul 24 03:45:53.255265 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:45:53.255230 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8hlhr_041beb02-32e9-4b8e-8f9d-34ee8d1a72c6/global-pull-secret-syncer/0.log" Jul 24 03:45:53.445028 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:45:53.444996 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qmbvk_c2c5797b-2cf9-4743-b334-7efa2192095a/konnectivity-agent/0.log" Jul 24 03:45:53.489424 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:45:53.489388 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-134.ec2.internal_0ae97c5152448a4aac7688f6e2b6cf38/haproxy/0.log" Jul 24 03:45:58.644386 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:45:58.644355 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-vkx42_9c4703c1-46e3-4cfc-a3d2-9d9710fae811/kuadrant-console-plugin/0.log" Jul 24 03:45:58.826078 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:45:58.826042 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-x69rz_e016a7ba-4041-448e-a651-65655b698f6d/manager/0.log" Jul 24 03:46:00.864479 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:00.864443 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_be51bc2a-40bf-474d-8a89-cfadc92609a4/alertmanager/0.log" Jul 24 03:46:00.887776 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:00.887751 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_be51bc2a-40bf-474d-8a89-cfadc92609a4/config-reloader/0.log" Jul 24 03:46:00.911042 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:00.911012 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_be51bc2a-40bf-474d-8a89-cfadc92609a4/kube-rbac-proxy-web/0.log" Jul 24 03:46:00.944934 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:00.944913 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_be51bc2a-40bf-474d-8a89-cfadc92609a4/kube-rbac-proxy/0.log" Jul 24 03:46:00.967515 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:00.967488 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_be51bc2a-40bf-474d-8a89-cfadc92609a4/kube-rbac-proxy-metric/0.log" Jul 24 03:46:00.991148 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:00.991128 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_be51bc2a-40bf-474d-8a89-cfadc92609a4/prom-label-proxy/0.log" Jul 24 03:46:01.020343 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:01.020320 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_be51bc2a-40bf-474d-8a89-cfadc92609a4/init-config-reloader/0.log" Jul 24 03:46:01.292439 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:01.292359 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-d4b79fc88-njng4_b3f0b141-5296-41c1-afd2-649194830051/metrics-server/0.log" Jul 24 03:46:01.566972 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:01.566874 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hkbh6_28810c1c-0d61-437d-b68d-79d6c933120c/node-exporter/0.log" Jul 24 03:46:01.627572 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:01.627544 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hkbh6_28810c1c-0d61-437d-b68d-79d6c933120c/kube-rbac-proxy/0.log" Jul 24 03:46:01.675615 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:01.675587 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hkbh6_28810c1c-0d61-437d-b68d-79d6c933120c/init-textfile/0.log" Jul 24 03:46:01.828450 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:01.828411 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-sgn88_3ae74270-00fd-466b-9359-389661481fff/kube-rbac-proxy-main/0.log" Jul 24 03:46:01.855773 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:01.855744 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-sgn88_3ae74270-00fd-466b-9359-389661481fff/kube-rbac-proxy-self/0.log" Jul 24 03:46:01.879170 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:01.879144 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-sgn88_3ae74270-00fd-466b-9359-389661481fff/openshift-state-metrics/0.log" Jul 24 03:46:01.915110 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:01.915070 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_348f762b-b7d0-4be5-af57-41b58d6140f8/prometheus/0.log" Jul 24 03:46:01.937739 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:01.937713 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_348f762b-b7d0-4be5-af57-41b58d6140f8/config-reloader/0.log" Jul 24 03:46:01.960930 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:01.960905 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_348f762b-b7d0-4be5-af57-41b58d6140f8/thanos-sidecar/0.log" Jul 24 03:46:01.984678 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:01.984633 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_348f762b-b7d0-4be5-af57-41b58d6140f8/kube-rbac-proxy-web/0.log" Jul 24 03:46:02.012643 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:02.012617 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_348f762b-b7d0-4be5-af57-41b58d6140f8/kube-rbac-proxy/0.log" Jul 24 03:46:02.050437 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:02.050405 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_348f762b-b7d0-4be5-af57-41b58d6140f8/kube-rbac-proxy-thanos/0.log" Jul 24 03:46:02.077193 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:02.077169 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_348f762b-b7d0-4be5-af57-41b58d6140f8/init-config-reloader/0.log" Jul 24 03:46:02.108782 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:02.108691 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-qsrcp_27c2b435-24ce-4119-a81f-3aae0aa7d911/prometheus-operator/0.log" Jul 24 03:46:02.134644 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:02.134619 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-qsrcp_27c2b435-24ce-4119-a81f-3aae0aa7d911/kube-rbac-proxy/0.log" Jul 24 03:46:02.169741 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:02.169716 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-ts9wj_b4a79ed4-fe4a-407b-b0de-88c3206cd524/prometheus-operator-admission-webhook/0.log" Jul 24 03:46:02.996260 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:02.996228 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-f2589/perf-node-gather-daemonset-gl55v"] Jul 24 03:46:02.996625 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:02.996574 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28a198b9-a479-4154-8ed1-60d9c3ffa0fc" containerName="cleanup" Jul 24 03:46:02.996625 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:02.996586 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="28a198b9-a479-4154-8ed1-60d9c3ffa0fc" containerName="cleanup" Jul 24 03:46:02.996728 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:02.996652 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="28a198b9-a479-4154-8ed1-60d9c3ffa0fc" containerName="cleanup" Jul 24 03:46:02.999895 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:02.999879 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f2589/perf-node-gather-daemonset-gl55v" Jul 24 03:46:03.014009 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:03.013988 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-f2589\"/\"kube-root-ca.crt\"" Jul 24 03:46:03.014480 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:03.014462 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-f2589\"/\"default-dockercfg-294hf\"" Jul 24 03:46:03.014607 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:03.014591 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-f2589\"/\"openshift-service-ca.crt\"" Jul 24 03:46:03.045988 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:03.045956 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jx9ps\" (UniqueName: \"kubernetes.io/projected/0839d31f-29a7-4d2b-9a5a-b60fb1100fcf-kube-api-access-jx9ps\") pod \"perf-node-gather-daemonset-gl55v\" (UID: \"0839d31f-29a7-4d2b-9a5a-b60fb1100fcf\") " pod="openshift-must-gather-f2589/perf-node-gather-daemonset-gl55v" Jul 24 03:46:03.046190 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:03.045993 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0839d31f-29a7-4d2b-9a5a-b60fb1100fcf-sys\") pod \"perf-node-gather-daemonset-gl55v\" (UID: \"0839d31f-29a7-4d2b-9a5a-b60fb1100fcf\") " pod="openshift-must-gather-f2589/perf-node-gather-daemonset-gl55v" Jul 24 03:46:03.046190 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:03.046021 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0839d31f-29a7-4d2b-9a5a-b60fb1100fcf-proc\") pod \"perf-node-gather-daemonset-gl55v\" (UID: \"0839d31f-29a7-4d2b-9a5a-b60fb1100fcf\") " pod="openshift-must-gather-f2589/perf-node-gather-daemonset-gl55v" Jul 24 03:46:03.046190 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:03.046076 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0839d31f-29a7-4d2b-9a5a-b60fb1100fcf-podres\") pod \"perf-node-gather-daemonset-gl55v\" (UID: \"0839d31f-29a7-4d2b-9a5a-b60fb1100fcf\") " pod="openshift-must-gather-f2589/perf-node-gather-daemonset-gl55v" Jul 24 03:46:03.046190 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:03.046148 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0839d31f-29a7-4d2b-9a5a-b60fb1100fcf-lib-modules\") pod \"perf-node-gather-daemonset-gl55v\" (UID: \"0839d31f-29a7-4d2b-9a5a-b60fb1100fcf\") " pod="openshift-must-gather-f2589/perf-node-gather-daemonset-gl55v" Jul 24 03:46:03.047148 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:03.047124 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-f2589/perf-node-gather-daemonset-gl55v"] Jul 24 03:46:03.147494 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:03.147455 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0839d31f-29a7-4d2b-9a5a-b60fb1100fcf-lib-modules\") pod \"perf-node-gather-daemonset-gl55v\" (UID: \"0839d31f-29a7-4d2b-9a5a-b60fb1100fcf\") " pod="openshift-must-gather-f2589/perf-node-gather-daemonset-gl55v" Jul 24 03:46:03.147729 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:03.147518 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jx9ps\" (UniqueName: \"kubernetes.io/projected/0839d31f-29a7-4d2b-9a5a-b60fb1100fcf-kube-api-access-jx9ps\") pod \"perf-node-gather-daemonset-gl55v\" (UID: \"0839d31f-29a7-4d2b-9a5a-b60fb1100fcf\") " pod="openshift-must-gather-f2589/perf-node-gather-daemonset-gl55v" Jul 24 03:46:03.147729 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:03.147630 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0839d31f-29a7-4d2b-9a5a-b60fb1100fcf-sys\") pod \"perf-node-gather-daemonset-gl55v\" (UID: \"0839d31f-29a7-4d2b-9a5a-b60fb1100fcf\") " pod="openshift-must-gather-f2589/perf-node-gather-daemonset-gl55v" Jul 24 03:46:03.147729 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:03.147690 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0839d31f-29a7-4d2b-9a5a-b60fb1100fcf-proc\") pod \"perf-node-gather-daemonset-gl55v\" (UID: \"0839d31f-29a7-4d2b-9a5a-b60fb1100fcf\") " pod="openshift-must-gather-f2589/perf-node-gather-daemonset-gl55v" Jul 24 03:46:03.147729 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:03.147699 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0839d31f-29a7-4d2b-9a5a-b60fb1100fcf-lib-modules\") pod \"perf-node-gather-daemonset-gl55v\" (UID: \"0839d31f-29a7-4d2b-9a5a-b60fb1100fcf\") " pod="openshift-must-gather-f2589/perf-node-gather-daemonset-gl55v" Jul 24 03:46:03.147943 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:03.147742 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0839d31f-29a7-4d2b-9a5a-b60fb1100fcf-sys\") pod \"perf-node-gather-daemonset-gl55v\" (UID: \"0839d31f-29a7-4d2b-9a5a-b60fb1100fcf\") " pod="openshift-must-gather-f2589/perf-node-gather-daemonset-gl55v" Jul 24 03:46:03.147943 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:03.147753 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0839d31f-29a7-4d2b-9a5a-b60fb1100fcf-proc\") pod \"perf-node-gather-daemonset-gl55v\" (UID: \"0839d31f-29a7-4d2b-9a5a-b60fb1100fcf\") " pod="openshift-must-gather-f2589/perf-node-gather-daemonset-gl55v" Jul 24 03:46:03.147943 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:03.147766 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0839d31f-29a7-4d2b-9a5a-b60fb1100fcf-podres\") pod \"perf-node-gather-daemonset-gl55v\" (UID: \"0839d31f-29a7-4d2b-9a5a-b60fb1100fcf\") " pod="openshift-must-gather-f2589/perf-node-gather-daemonset-gl55v" Jul 24 03:46:03.147943 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:03.147849 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0839d31f-29a7-4d2b-9a5a-b60fb1100fcf-podres\") pod \"perf-node-gather-daemonset-gl55v\" (UID: \"0839d31f-29a7-4d2b-9a5a-b60fb1100fcf\") " pod="openshift-must-gather-f2589/perf-node-gather-daemonset-gl55v" Jul 24 03:46:03.157275 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:03.157256 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jx9ps\" (UniqueName: \"kubernetes.io/projected/0839d31f-29a7-4d2b-9a5a-b60fb1100fcf-kube-api-access-jx9ps\") pod \"perf-node-gather-daemonset-gl55v\" (UID: \"0839d31f-29a7-4d2b-9a5a-b60fb1100fcf\") " pod="openshift-must-gather-f2589/perf-node-gather-daemonset-gl55v" Jul 24 03:46:03.309647 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:03.309548 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f2589/perf-node-gather-daemonset-gl55v" Jul 24 03:46:03.449313 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:03.449280 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-f2589/perf-node-gather-daemonset-gl55v"] Jul 24 03:46:03.452451 ip-10-0-139-134 kubenswrapper[2579]: W0724 03:46:03.452421 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0839d31f_29a7_4d2b_9a5a_b60fb1100fcf.slice/crio-d4b415216636633452adc22ea8d4a76c8794ef087796ec3c334a40d6588fc3c9 WatchSource:0}: Error finding container d4b415216636633452adc22ea8d4a76c8794ef087796ec3c334a40d6588fc3c9: Status 404 returned error can't find the container with id d4b415216636633452adc22ea8d4a76c8794ef087796ec3c334a40d6588fc3c9 Jul 24 03:46:04.097225 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:04.097186 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f2589/perf-node-gather-daemonset-gl55v" event={"ID":"0839d31f-29a7-4d2b-9a5a-b60fb1100fcf","Type":"ContainerStarted","Data":"2958a6f4066a8928060b36bf2fbdc362553af318a68cbc45cd0310f2ee221481"} Jul 24 03:46:04.097628 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:04.097270 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f2589/perf-node-gather-daemonset-gl55v" event={"ID":"0839d31f-29a7-4d2b-9a5a-b60fb1100fcf","Type":"ContainerStarted","Data":"d4b415216636633452adc22ea8d4a76c8794ef087796ec3c334a40d6588fc3c9"} Jul 24 03:46:04.097628 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:04.097312 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-f2589/perf-node-gather-daemonset-gl55v" Jul 24 03:46:04.129861 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:04.129808 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-f2589/perf-node-gather-daemonset-gl55v" podStartSLOduration=2.129791505 podStartE2EDuration="2.129791505s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 03:46:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:46:04.127332381 +0000 UTC m=+1661.402903731" watchObservedRunningTime="2026-07-24 03:46:04.129791505 +0000 UTC m=+1661.405362855" Jul 24 03:46:05.350545 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:05.350516 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-66967c796d-tq4w5_bde7a761-9336-4804-9891-9a8c0ca5ea3c/console/0.log" Jul 24 03:46:05.411419 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:05.411390 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-9h4l2_18fbf1b4-55a4-4e8c-806e-408fb712a12a/download-server/0.log" Jul 24 03:46:07.140716 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:07.140657 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rbhtr_642b4433-a01d-40ce-a14c-7dd42ab4e837/dns/0.log" Jul 24 03:46:07.175077 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:07.175048 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rbhtr_642b4433-a01d-40ce-a14c-7dd42ab4e837/kube-rbac-proxy/0.log" Jul 24 03:46:07.200897 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:07.200872 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gfdrz_5fc751d2-6a75-413f-881c-5a3c92935de3/dns-node-resolver/0.log" Jul 24 03:46:07.839032 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:07.839002 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5nlhv_46b3c291-be47-4a57-82c2-b82e84a9909f/node-ca/0.log" Jul 24 03:46:08.982246 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:08.982208 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7ccbc5c95c-n9qf9_a84843d1-19b7-468e-ac6f-f925ed893a36/kube-auth-proxy/0.log" Jul 24 03:46:09.824785 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:09.824751 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5zwrl_2e237dac-e8fc-4d7c-a87c-5c73b5bc1091/serve-healthcheck-canary/0.log" Jul 24 03:46:10.112699 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:10.112597 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-f2589/perf-node-gather-daemonset-gl55v" Jul 24 03:46:10.368962 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:10.368871 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5t6sr_cf923b24-c681-4393-b0a9-7db5c0183552/kube-rbac-proxy/0.log" Jul 24 03:46:10.392454 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:10.392433 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5t6sr_cf923b24-c681-4393-b0a9-7db5c0183552/exporter/0.log" Jul 24 03:46:10.413145 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:10.413123 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5t6sr_cf923b24-c681-4393-b0a9-7db5c0183552/extractor/0.log" Jul 24 03:46:13.364612 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:13.364585 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c75c644cf-j7hn4_b56fa8eb-8fb8-4dea-a80d-5e9dee75941e/manager/0.log" Jul 24 03:46:13.383354 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:13.383329 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c75c644cf-j7hn4_b56fa8eb-8fb8-4dea-a80d-5e9dee75941e/copy-manifests/0.log" Jul 24 03:46:13.954707 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:13.954660 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7ccbc5c95c-n9qf9_a84843d1-19b7-468e-ac6f-f925ed893a36/kube-auth-proxy/0.log" Jul 24 03:46:14.805132 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:14.805104 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-df5f88749-znv82_ada0295b-a707-4100-8d0f-9bf799c9fdc4/manager/0.log" Jul 24 03:46:15.314637 ip-10-0-139-134 kubenswrapper[2579]: I0724 03:46:15.314605 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7ccbc5c95c-n9qf9_a84843d1-19b7-468e-ac6f-f925ed893a36/kube-auth-proxy/0.log"