Jul 23 04:37:19.919971 ip-10-0-128-96 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 23 04:37:19.920026 ip-10-0-128-96 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 23 04:37:19.920033 ip-10-0-128-96 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 23 04:37:19.920246 ip-10-0-128-96 systemd[1]: Failed to start Kubernetes Kubelet. Jul 23 04:37:30.020988 ip-10-0-128-96 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 23 04:37:30.021010 ip-10-0-128-96 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 482fff4d60114a0db4cdda52b2509fec -- Jul 23 04:39:23.239415 ip-10-0-128-96 systemd[1]: Starting Kubernetes Kubelet... Jul 23 04:39:23.662378 ip-10-0-128-96 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 04:39:23.662378 ip-10-0-128-96 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 23 04:39:23.662378 ip-10-0-128-96 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 04:39:23.662378 ip-10-0-128-96 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 23 04:39:23.662378 ip-10-0-128-96 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 04:39:23.663051 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.662451 2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 23 04:39:23.666671 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666653 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 04:39:23.666671 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666670 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 04:39:23.666782 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666674 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 04:39:23.666782 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666677 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 04:39:23.666782 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666680 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 04:39:23.666782 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666683 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 04:39:23.666782 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666686 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 04:39:23.666782 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666688 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 04:39:23.666782 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666692 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 04:39:23.666782 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666695 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 04:39:23.666782 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666698 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 04:39:23.666782 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666700 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 04:39:23.666782 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666703 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 04:39:23.666782 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666705 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 04:39:23.666782 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666708 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 04:39:23.666782 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666712 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 04:39:23.666782 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666716 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 04:39:23.666782 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666719 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 04:39:23.666782 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666722 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 04:39:23.666782 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666724 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 04:39:23.666782 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666727 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 04:39:23.667256 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666730 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 04:39:23.667256 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666732 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 04:39:23.667256 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666735 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 04:39:23.667256 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666738 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 04:39:23.667256 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666740 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 04:39:23.667256 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666748 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 04:39:23.667256 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666750 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 04:39:23.667256 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666753 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 04:39:23.667256 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666756 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 04:39:23.667256 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666758 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 04:39:23.667256 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666761 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 04:39:23.667256 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666763 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 04:39:23.667256 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666766 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 04:39:23.667256 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666768 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 04:39:23.667256 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666771 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 04:39:23.667256 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666774 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 04:39:23.667256 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666776 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 04:39:23.667256 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666779 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 04:39:23.667256 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666783 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 04:39:23.667256 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666785 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 04:39:23.667727 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666788 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 04:39:23.667727 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666791 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 04:39:23.667727 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666793 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 04:39:23.667727 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666796 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 04:39:23.667727 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666799 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 04:39:23.667727 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666801 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 04:39:23.667727 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666804 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 04:39:23.667727 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666807 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 04:39:23.667727 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666809 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 04:39:23.667727 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666811 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 04:39:23.667727 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666814 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 23 04:39:23.667727 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666816 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 04:39:23.667727 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666819 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 04:39:23.667727 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666822 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 04:39:23.667727 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666824 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 04:39:23.667727 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666826 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 04:39:23.667727 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666829 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 04:39:23.667727 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666832 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 04:39:23.667727 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666834 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 04:39:23.667727 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666837 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 04:39:23.668267 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666839 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 04:39:23.668267 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666842 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 04:39:23.668267 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666844 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 04:39:23.668267 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666847 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 04:39:23.668267 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666849 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 04:39:23.668267 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666851 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 04:39:23.668267 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666856 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 04:39:23.668267 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666858 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 04:39:23.668267 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666861 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 04:39:23.668267 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666863 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 04:39:23.668267 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666866 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 04:39:23.668267 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666868 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 04:39:23.668267 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666873 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 04:39:23.668267 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666876 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 04:39:23.668267 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666881 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 04:39:23.668267 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666883 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 04:39:23.668267 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666887 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 04:39:23.668267 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666891 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 04:39:23.668267 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666893 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 04:39:23.668716 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666896 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 04:39:23.668716 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666898 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 04:39:23.668716 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666901 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 04:39:23.668716 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666904 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 04:39:23.668716 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666906 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 04:39:23.668716 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666909 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 04:39:23.668716 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666911 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 04:39:23.668716 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666914 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 04:39:23.668716 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666916 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 04:39:23.668716 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666919 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 04:39:23.668716 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666921 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 04:39:23.668716 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666924 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 04:39:23.668716 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666926 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 04:39:23.668716 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666928 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 04:39:23.668716 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666931 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 04:39:23.668716 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666934 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 04:39:23.668716 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666936 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 04:39:23.668716 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666938 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 04:39:23.668716 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.666941 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 04:39:23.669179 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667651 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 04:39:23.669179 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667658 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 04:39:23.669179 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667661 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 04:39:23.669179 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667663 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 04:39:23.669179 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667666 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 04:39:23.669179 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667669 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 04:39:23.669179 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667671 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 04:39:23.669179 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667674 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 04:39:23.669179 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667676 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 04:39:23.669179 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667679 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 04:39:23.669179 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667681 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 04:39:23.669179 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667685 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 04:39:23.669179 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667687 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 04:39:23.669179 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667690 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 04:39:23.669179 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667692 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 04:39:23.669179 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667695 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 04:39:23.669179 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667698 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 04:39:23.669179 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667700 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 04:39:23.669179 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667703 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 04:39:23.669179 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667705 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 04:39:23.669675 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667708 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 04:39:23.669675 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667710 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 04:39:23.669675 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667712 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 04:39:23.669675 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667714 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 04:39:23.669675 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667717 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 04:39:23.669675 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667719 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 04:39:23.669675 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667722 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 04:39:23.669675 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667724 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 04:39:23.669675 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667726 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 04:39:23.669675 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667729 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 04:39:23.669675 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667731 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 04:39:23.669675 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667734 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 04:39:23.669675 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667737 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 04:39:23.669675 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667740 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 04:39:23.669675 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667743 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 04:39:23.669675 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667747 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 04:39:23.669675 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667750 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 04:39:23.669675 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667753 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 04:39:23.669675 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667756 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 04:39:23.669675 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667758 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 04:39:23.670165 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667762 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 04:39:23.670165 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667766 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 04:39:23.670165 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667768 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 04:39:23.670165 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667771 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 04:39:23.670165 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667784 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 04:39:23.670165 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667787 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 04:39:23.670165 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667789 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 04:39:23.670165 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667792 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 04:39:23.670165 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667794 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 04:39:23.670165 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667797 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 04:39:23.670165 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667799 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 04:39:23.670165 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667802 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 04:39:23.670165 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667804 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 04:39:23.670165 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667807 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 04:39:23.670165 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667809 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 04:39:23.670165 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667812 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 04:39:23.670165 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667815 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 04:39:23.670165 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667817 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 04:39:23.670165 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667820 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 04:39:23.670659 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667822 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 04:39:23.670659 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667824 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 04:39:23.670659 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667827 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 04:39:23.670659 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667830 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 04:39:23.670659 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667832 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 04:39:23.670659 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667835 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 04:39:23.670659 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667838 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 04:39:23.670659 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667840 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 04:39:23.670659 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667843 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 04:39:23.670659 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667846 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 04:39:23.670659 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667848 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 04:39:23.670659 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667851 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 04:39:23.670659 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667853 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 04:39:23.670659 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667856 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 04:39:23.670659 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667858 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 23 04:39:23.670659 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667860 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 04:39:23.670659 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667863 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 04:39:23.670659 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667866 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 04:39:23.670659 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667869 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 04:39:23.670659 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667872 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 04:39:23.671160 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667874 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 04:39:23.671160 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667876 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 04:39:23.671160 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667879 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 04:39:23.671160 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667881 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 04:39:23.671160 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667891 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 04:39:23.671160 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667894 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 04:39:23.671160 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667899 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 04:39:23.671160 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667902 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 04:39:23.671160 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667906 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 04:39:23.671160 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667909 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 04:39:23.671160 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667912 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 04:39:23.671160 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667915 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 04:39:23.671160 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667917 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 04:39:23.671160 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667919 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 04:39:23.671160 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667925 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 04:39:23.671160 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667928 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 04:39:23.671160 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667931 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 04:39:23.671160 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667934 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 04:39:23.671160 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667936 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 04:39:23.671160 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.667939 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 04:39:23.671733 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668327 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 04:39:23.671733 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668332 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 04:39:23.671733 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668335 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 04:39:23.671733 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668338 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 04:39:23.671733 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668341 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 04:39:23.671733 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668343 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 04:39:23.671733 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668346 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 04:39:23.671733 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668348 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 04:39:23.671733 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668351 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 04:39:23.671733 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668353 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 04:39:23.671733 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668357 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 04:39:23.671733 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668360 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 04:39:23.671733 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668362 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 04:39:23.671733 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668364 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 04:39:23.671733 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668369 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 04:39:23.671733 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668373 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 04:39:23.671733 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668375 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 04:39:23.671733 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668378 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 04:39:23.671733 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668381 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 04:39:23.671733 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668384 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 04:39:23.672246 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668386 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 04:39:23.672246 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668389 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 04:39:23.672246 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668392 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 04:39:23.672246 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668394 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 04:39:23.672246 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668396 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 04:39:23.672246 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668399 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 04:39:23.672246 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668403 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 04:39:23.672246 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668406 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 04:39:23.672246 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668408 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 04:39:23.672246 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668411 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 04:39:23.672246 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668414 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 04:39:23.672246 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668416 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 04:39:23.672246 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668419 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 04:39:23.672246 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668421 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 04:39:23.672246 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668423 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 04:39:23.672246 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668426 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 04:39:23.672246 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668429 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 04:39:23.672246 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668433 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 23 04:39:23.672246 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668435 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 04:39:23.672704 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668438 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 04:39:23.672704 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668440 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 04:39:23.672704 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668443 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 04:39:23.672704 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668445 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 04:39:23.672704 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668448 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 04:39:23.672704 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668451 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 04:39:23.672704 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668454 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 04:39:23.672704 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668456 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 04:39:23.672704 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668458 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 04:39:23.672704 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668461 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 04:39:23.672704 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668463 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 04:39:23.672704 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668465 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 04:39:23.672704 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668468 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 04:39:23.672704 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668472 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 04:39:23.672704 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668474 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 04:39:23.672704 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668477 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 04:39:23.672704 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668479 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 04:39:23.672704 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668482 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 04:39:23.672704 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668484 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 04:39:23.672704 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668488 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 04:39:23.673188 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668490 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 04:39:23.673188 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668493 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 04:39:23.673188 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668495 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 04:39:23.673188 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668498 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 04:39:23.673188 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668501 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 04:39:23.673188 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668503 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 04:39:23.673188 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668506 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 04:39:23.673188 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668508 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 04:39:23.673188 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668511 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 04:39:23.673188 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668513 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 04:39:23.673188 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668516 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 04:39:23.673188 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668518 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 04:39:23.673188 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668520 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 04:39:23.673188 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668523 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 04:39:23.673188 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668525 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 04:39:23.673188 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668527 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 04:39:23.673188 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668530 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 04:39:23.673188 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668533 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 04:39:23.673188 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668536 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 04:39:23.673188 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668538 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 04:39:23.673666 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668541 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 04:39:23.673666 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668543 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 04:39:23.673666 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668545 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 04:39:23.673666 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668548 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 04:39:23.673666 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668550 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 04:39:23.673666 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668554 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 04:39:23.673666 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668556 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 04:39:23.673666 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668559 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 04:39:23.673666 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668561 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 04:39:23.673666 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668564 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 04:39:23.673666 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668567 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 04:39:23.673666 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668571 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 04:39:23.673666 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668574 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 04:39:23.673666 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668577 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 04:39:23.673666 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668580 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 04:39:23.673666 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668582 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 04:39:23.673666 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668584 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 04:39:23.673666 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668587 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 04:39:23.673666 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668589 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 04:39:23.674121 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.668592 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 04:39:23.674121 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668662 2562 flags.go:64] FLAG: --address="0.0.0.0" Jul 23 04:39:23.674121 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668674 2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 23 04:39:23.674121 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668680 2562 flags.go:64] FLAG: --anonymous-auth="true" Jul 23 04:39:23.674121 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668684 2562 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 23 04:39:23.674121 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668689 2562 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 23 04:39:23.674121 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668692 2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 23 04:39:23.674121 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668696 2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 23 04:39:23.674121 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668700 2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 23 04:39:23.674121 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668703 2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 23 04:39:23.674121 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668706 2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 23 04:39:23.674121 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668710 2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 23 04:39:23.674121 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668713 2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 23 04:39:23.674121 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668716 2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 23 04:39:23.674121 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668719 2562 flags.go:64] FLAG: --cgroup-root="" Jul 23 04:39:23.674121 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668722 2562 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 23 04:39:23.674121 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668725 2562 flags.go:64] FLAG: --client-ca-file="" Jul 23 04:39:23.674121 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668728 2562 flags.go:64] FLAG: --cloud-provider="external" Jul 23 04:39:23.674121 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668731 2562 flags.go:64] FLAG: --cluster-dns="[]" Jul 23 04:39:23.674121 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668735 2562 flags.go:64] FLAG: --cluster-domain="" Jul 23 04:39:23.674121 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668738 2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 23 04:39:23.674121 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668741 2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 23 04:39:23.674121 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668743 2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 23 04:39:23.674121 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668747 2562 flags.go:64] FLAG: --container-log-max-files="5" Jul 23 04:39:23.674700 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668751 2562 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 23 04:39:23.674700 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668754 2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 23 04:39:23.674700 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668757 2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 23 04:39:23.674700 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668760 2562 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 23 04:39:23.674700 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668763 2562 flags.go:64] FLAG: --contention-profiling="false" Jul 23 04:39:23.674700 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668766 2562 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 23 04:39:23.674700 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668768 2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 23 04:39:23.674700 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668771 2562 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 23 04:39:23.674700 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668774 2562 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 23 04:39:23.674700 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668778 2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 23 04:39:23.674700 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668780 2562 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 23 04:39:23.674700 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668783 2562 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 23 04:39:23.674700 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668786 2562 flags.go:64] FLAG: --enable-load-reader="false" Jul 23 04:39:23.674700 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668788 2562 flags.go:64] FLAG: --enable-server="true" Jul 23 04:39:23.674700 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668791 2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 23 04:39:23.674700 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668795 2562 flags.go:64] FLAG: --event-burst="100" Jul 23 04:39:23.674700 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668798 2562 flags.go:64] FLAG: --event-qps="50" Jul 23 04:39:23.674700 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668801 2562 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 23 04:39:23.674700 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668804 2562 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 23 04:39:23.674700 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668807 2562 flags.go:64] FLAG: --eviction-hard="" Jul 23 04:39:23.674700 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668811 2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 23 04:39:23.674700 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668814 2562 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 23 04:39:23.674700 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668816 2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 23 04:39:23.674700 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668819 2562 flags.go:64] FLAG: --eviction-soft="" Jul 23 04:39:23.674700 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668822 2562 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 23 04:39:23.675313 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668824 2562 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 23 04:39:23.675313 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668827 2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 23 04:39:23.675313 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668830 2562 flags.go:64] FLAG: --experimental-mounter-path="" Jul 23 04:39:23.675313 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668832 2562 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 23 04:39:23.675313 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668835 2562 flags.go:64] FLAG: --fail-swap-on="true" Jul 23 04:39:23.675313 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668838 2562 flags.go:64] FLAG: --feature-gates="" Jul 23 04:39:23.675313 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668841 2562 flags.go:64] FLAG: --file-check-frequency="20s" Jul 23 04:39:23.675313 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668844 2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 23 04:39:23.675313 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668847 2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 23 04:39:23.675313 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668850 2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 23 04:39:23.675313 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668853 2562 flags.go:64] FLAG: --healthz-port="10248" Jul 23 04:39:23.675313 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668856 2562 flags.go:64] FLAG: --help="false" Jul 23 04:39:23.675313 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668859 2562 flags.go:64] FLAG: --hostname-override="ip-10-0-128-96.ec2.internal" Jul 23 04:39:23.675313 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668862 2562 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 23 04:39:23.675313 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668865 2562 flags.go:64] FLAG: --http-check-frequency="20s" Jul 23 04:39:23.675313 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668868 2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 23 04:39:23.675313 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668871 2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 23 04:39:23.675313 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668874 2562 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 23 04:39:23.675313 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668877 2562 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 23 04:39:23.675313 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668879 2562 flags.go:64] FLAG: --image-service-endpoint="" Jul 23 04:39:23.675313 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668882 2562 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 23 04:39:23.675313 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668885 2562 flags.go:64] FLAG: --kube-api-burst="100" Jul 23 04:39:23.675313 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668887 2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 23 04:39:23.676368 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668890 2562 flags.go:64] FLAG: --kube-api-qps="50" Jul 23 04:39:23.676368 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668893 2562 flags.go:64] FLAG: --kube-reserved="" Jul 23 04:39:23.676368 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668895 2562 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 23 04:39:23.676368 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668898 2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 23 04:39:23.676368 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668901 2562 flags.go:64] FLAG: --kubelet-cgroups="" Jul 23 04:39:23.676368 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668904 2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 23 04:39:23.676368 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668907 2562 flags.go:64] FLAG: --lock-file="" Jul 23 04:39:23.676368 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668910 2562 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 23 04:39:23.676368 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668912 2562 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 23 04:39:23.676368 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668915 2562 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 23 04:39:23.676368 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668920 2562 flags.go:64] FLAG: --log-json-split-stream="false" Jul 23 04:39:23.676368 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668923 2562 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 23 04:39:23.676368 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668925 2562 flags.go:64] FLAG: --log-text-split-stream="false" Jul 23 04:39:23.676368 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668928 2562 flags.go:64] FLAG: --logging-format="text" Jul 23 04:39:23.676368 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668931 2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 23 04:39:23.676368 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668934 2562 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 23 04:39:23.676368 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668937 2562 flags.go:64] FLAG: --manifest-url="" Jul 23 04:39:23.676368 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668940 2562 flags.go:64] FLAG: --manifest-url-header="" Jul 23 04:39:23.676368 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668944 2562 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 23 04:39:23.676368 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668946 2562 flags.go:64] FLAG: --max-open-files="1000000" Jul 23 04:39:23.676368 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668951 2562 flags.go:64] FLAG: --max-pods="110" Jul 23 04:39:23.676368 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668953 2562 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 23 04:39:23.676368 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668956 2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 23 04:39:23.676368 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668959 2562 flags.go:64] FLAG: --memory-manager-policy="None" Jul 23 04:39:23.676368 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668961 2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 23 04:39:23.677443 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668964 2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 23 04:39:23.677443 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668967 2562 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 23 04:39:23.677443 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668969 2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 23 04:39:23.677443 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668976 2562 flags.go:64] FLAG: --node-status-max-images="50" Jul 23 04:39:23.677443 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668979 2562 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 23 04:39:23.677443 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668982 2562 flags.go:64] FLAG: --oom-score-adj="-999" Jul 23 04:39:23.677443 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668984 2562 flags.go:64] FLAG: --pod-cidr="" Jul 23 04:39:23.677443 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668987 2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 23 04:39:23.677443 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668991 2562 flags.go:64] FLAG: --pod-manifest-path="" Jul 23 04:39:23.677443 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668994 2562 flags.go:64] FLAG: --pod-max-pids="-1" Jul 23 04:39:23.677443 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668997 2562 flags.go:64] FLAG: --pods-per-core="0" Jul 23 04:39:23.677443 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.668999 2562 flags.go:64] FLAG: --port="10250" Jul 23 04:39:23.677443 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669002 2562 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 23 04:39:23.677443 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669005 2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0c0b00cab4614379f" Jul 23 04:39:23.677443 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669008 2562 flags.go:64] FLAG: --qos-reserved="" Jul 23 04:39:23.677443 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669011 2562 flags.go:64] FLAG: --read-only-port="10255" Jul 23 04:39:23.677443 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669014 2562 flags.go:64] FLAG: --register-node="true" Jul 23 04:39:23.677443 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669017 2562 flags.go:64] FLAG: --register-with-taints="" Jul 23 04:39:23.677443 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669020 2562 flags.go:64] FLAG: --registry-burst="10" Jul 23 04:39:23.677443 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669023 2562 flags.go:64] FLAG: --registry-qps="5" Jul 23 04:39:23.677443 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669025 2562 flags.go:64] FLAG: --reserved-cpus="" Jul 23 04:39:23.677443 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669028 2562 flags.go:64] FLAG: --reserved-memory="" Jul 23 04:39:23.677443 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669031 2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 23 04:39:23.677443 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669034 2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 23 04:39:23.677443 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669037 2562 flags.go:64] FLAG: --rotate-certificates="false" Jul 23 04:39:23.678528 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669040 2562 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 23 04:39:23.678528 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669042 2562 flags.go:64] FLAG: --runonce="false" Jul 23 04:39:23.678528 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669045 2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 23 04:39:23.678528 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669048 2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 23 04:39:23.678528 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669050 2562 flags.go:64] FLAG: --seccomp-default="false" Jul 23 04:39:23.678528 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669053 2562 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 23 04:39:23.678528 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669056 2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 23 04:39:23.678528 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669059 2562 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 23 04:39:23.678528 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669062 2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 23 04:39:23.678528 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669064 2562 flags.go:64] FLAG: --storage-driver-password="root" Jul 23 04:39:23.678528 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669067 2562 flags.go:64] FLAG: --storage-driver-secure="false" Jul 23 04:39:23.678528 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669070 2562 flags.go:64] FLAG: --storage-driver-table="stats" Jul 23 04:39:23.678528 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669072 2562 flags.go:64] FLAG: --storage-driver-user="root" Jul 23 04:39:23.678528 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669075 2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 23 04:39:23.678528 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669077 2562 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 23 04:39:23.678528 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669080 2562 flags.go:64] FLAG: --system-cgroups="" Jul 23 04:39:23.678528 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669083 2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 23 04:39:23.678528 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669088 2562 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 23 04:39:23.678528 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669092 2562 flags.go:64] FLAG: --tls-cert-file="" Jul 23 04:39:23.678528 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669094 2562 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 23 04:39:23.678528 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669098 2562 flags.go:64] FLAG: --tls-min-version="" Jul 23 04:39:23.678528 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669101 2562 flags.go:64] FLAG: --tls-private-key-file="" Jul 23 04:39:23.678528 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669104 2562 flags.go:64] FLAG: --topology-manager-policy="none" Jul 23 04:39:23.678528 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669107 2562 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 23 04:39:23.678528 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669123 2562 flags.go:64] FLAG: --topology-manager-scope="container" Jul 23 04:39:23.679432 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669126 2562 flags.go:64] FLAG: --v="2" Jul 23 04:39:23.679432 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669130 2562 flags.go:64] FLAG: --version="false" Jul 23 04:39:23.679432 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669134 2562 flags.go:64] FLAG: --vmodule="" Jul 23 04:39:23.679432 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669137 2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 23 04:39:23.679432 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.669141 2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 23 04:39:23.679432 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669253 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 04:39:23.679432 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669258 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 04:39:23.679432 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669261 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 04:39:23.679432 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669264 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 04:39:23.679432 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669266 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 04:39:23.679432 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669269 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 04:39:23.679432 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669272 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 04:39:23.679432 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669274 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 04:39:23.679432 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669277 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 04:39:23.679432 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669280 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 04:39:23.679432 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669282 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 04:39:23.679432 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669284 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 04:39:23.679432 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669287 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 04:39:23.679432 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669289 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 04:39:23.679432 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669292 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 04:39:23.679432 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669294 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 04:39:23.680177 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669296 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 04:39:23.680177 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669299 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 04:39:23.680177 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669301 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 04:39:23.680177 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669304 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 04:39:23.680177 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669307 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 04:39:23.680177 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669310 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 04:39:23.680177 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669312 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 04:39:23.680177 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669315 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 04:39:23.680177 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669318 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 04:39:23.680177 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669321 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 04:39:23.680177 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669323 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 04:39:23.680177 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669327 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 04:39:23.680177 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669330 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 04:39:23.680177 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669333 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 04:39:23.680177 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669336 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 04:39:23.680177 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669338 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 04:39:23.680177 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669340 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 04:39:23.680177 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669344 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 04:39:23.680177 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669346 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 04:39:23.680177 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669348 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 04:39:23.681014 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669351 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 04:39:23.681014 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669353 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 23 04:39:23.681014 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669356 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 04:39:23.681014 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669358 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 04:39:23.681014 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669362 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 04:39:23.681014 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669367 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 04:39:23.681014 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669370 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 04:39:23.681014 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669373 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 04:39:23.681014 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669375 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 04:39:23.681014 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669378 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 04:39:23.681014 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669380 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 04:39:23.681014 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669383 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 04:39:23.681014 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669385 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 04:39:23.681014 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669387 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 04:39:23.681014 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669390 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 04:39:23.681014 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669392 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 04:39:23.681014 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669396 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 04:39:23.681014 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669399 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 04:39:23.681014 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669401 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 04:39:23.681014 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669404 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 04:39:23.681591 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669406 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 04:39:23.681591 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669409 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 04:39:23.681591 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669411 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 04:39:23.681591 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669414 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 04:39:23.681591 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669416 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 04:39:23.681591 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669419 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 04:39:23.681591 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669421 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 04:39:23.681591 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669424 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 04:39:23.681591 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669426 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 04:39:23.681591 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669429 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 04:39:23.681591 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669432 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 04:39:23.681591 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669434 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 04:39:23.681591 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669437 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 04:39:23.681591 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669439 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 04:39:23.681591 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669442 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 04:39:23.681591 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669444 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 04:39:23.681591 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669447 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 04:39:23.681591 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669449 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 04:39:23.681591 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669451 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 04:39:23.681591 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669454 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 04:39:23.682247 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669456 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 04:39:23.682247 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669459 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 04:39:23.682247 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669462 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 04:39:23.682247 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669464 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 04:39:23.682247 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669468 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 04:39:23.682247 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669471 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 04:39:23.682247 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669474 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 04:39:23.682247 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669477 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 04:39:23.682247 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669481 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 04:39:23.682247 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669483 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 04:39:23.682247 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669486 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 04:39:23.682247 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669488 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 04:39:23.682247 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669490 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 04:39:23.682247 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669493 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 04:39:23.682247 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669496 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 04:39:23.682247 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669498 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 04:39:23.682247 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669500 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 04:39:23.682247 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669503 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 04:39:23.682247 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669505 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 04:39:23.682741 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669508 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 04:39:23.682741 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669510 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 04:39:23.682741 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669514 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 04:39:23.682741 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.669516 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 04:39:23.682741 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.670064 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 04:39:23.682741 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.677640 2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 23 04:39:23.682741 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.677660 2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 23 04:39:23.682741 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.677680 2562 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 23 04:39:23.682741 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.677689 2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 23 04:39:23.682741 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677772 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 04:39:23.682741 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677780 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 04:39:23.682741 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677784 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 04:39:23.682741 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677788 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 04:39:23.682741 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677793 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 04:39:23.682741 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677797 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 04:39:23.683178 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677802 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 04:39:23.683178 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677806 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 04:39:23.683178 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677811 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 04:39:23.683178 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677815 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 04:39:23.683178 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677819 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 04:39:23.683178 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677823 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 04:39:23.683178 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677828 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 04:39:23.683178 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677833 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 04:39:23.683178 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677837 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 04:39:23.683178 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677841 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 04:39:23.683178 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677845 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 04:39:23.683178 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677850 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 04:39:23.683178 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677854 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 04:39:23.683178 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677858 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 04:39:23.683178 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677863 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 04:39:23.683178 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677867 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 04:39:23.683178 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677871 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 04:39:23.683178 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677875 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 04:39:23.683178 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677879 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 04:39:23.683178 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677883 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 04:39:23.683662 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677887 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 04:39:23.683662 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677891 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 04:39:23.683662 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677895 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 04:39:23.683662 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677901 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 04:39:23.683662 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677905 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 04:39:23.683662 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677909 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 04:39:23.683662 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677913 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 04:39:23.683662 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677917 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 04:39:23.683662 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677921 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 04:39:23.683662 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677925 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 04:39:23.683662 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677930 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 04:39:23.683662 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677934 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 04:39:23.683662 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677938 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 23 04:39:23.683662 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677942 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 04:39:23.683662 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677946 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 04:39:23.683662 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677950 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 04:39:23.683662 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677954 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 04:39:23.683662 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677958 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 04:39:23.683662 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677962 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 04:39:23.683662 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677967 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 04:39:23.684234 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677971 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 04:39:23.684234 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677975 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 04:39:23.684234 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677979 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 04:39:23.684234 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677983 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 04:39:23.684234 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677987 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 04:39:23.684234 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677991 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 04:39:23.684234 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677995 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 04:39:23.684234 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.677999 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 04:39:23.684234 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678003 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 04:39:23.684234 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678008 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 04:39:23.684234 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678012 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 04:39:23.684234 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678016 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 04:39:23.684234 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678020 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 04:39:23.684234 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678025 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 04:39:23.684234 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678028 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 04:39:23.684234 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678034 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 04:39:23.684234 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678039 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 04:39:23.684234 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678043 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 04:39:23.684234 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678048 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 04:39:23.684234 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678053 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 04:39:23.684782 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678057 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 04:39:23.684782 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678061 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 04:39:23.684782 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678065 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 04:39:23.684782 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678069 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 04:39:23.684782 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678074 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 04:39:23.684782 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678078 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 04:39:23.684782 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678084 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 04:39:23.684782 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678090 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 04:39:23.684782 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678095 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 04:39:23.684782 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678100 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 04:39:23.684782 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678106 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 04:39:23.684782 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678131 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 04:39:23.684782 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678135 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 04:39:23.684782 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678140 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 04:39:23.684782 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678144 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 04:39:23.684782 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678151 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 04:39:23.684782 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678156 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 04:39:23.684782 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678161 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 04:39:23.685234 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678165 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 04:39:23.685234 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678169 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 04:39:23.685234 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678174 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 04:39:23.685234 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678178 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 04:39:23.685234 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678182 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 04:39:23.685234 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678186 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 04:39:23.685234 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678190 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 04:39:23.685234 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678194 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 04:39:23.685234 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678198 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 04:39:23.685234 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678203 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 04:39:23.685234 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678208 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 04:39:23.685234 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678214 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 04:39:23.685234 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678218 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 04:39:23.685234 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678222 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 04:39:23.685234 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678226 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 04:39:23.685654 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.678234 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 04:39:23.685654 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678415 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 04:39:23.685654 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678423 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 04:39:23.685654 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678428 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 04:39:23.685654 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678432 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 04:39:23.685654 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678437 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 04:39:23.685654 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678441 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 04:39:23.685654 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678445 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 04:39:23.685654 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678449 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 04:39:23.685654 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678453 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 04:39:23.685654 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678457 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 04:39:23.685654 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678461 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 04:39:23.685654 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678465 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 04:39:23.685654 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678470 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 04:39:23.685654 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678474 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 04:39:23.686028 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678478 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 04:39:23.686028 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678482 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 04:39:23.686028 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678486 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 04:39:23.686028 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678491 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 04:39:23.686028 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678494 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 04:39:23.686028 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678499 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 04:39:23.686028 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678503 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 04:39:23.686028 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678506 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 04:39:23.686028 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678510 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 04:39:23.686028 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678515 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 04:39:23.686028 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678519 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 04:39:23.686028 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678524 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 04:39:23.686028 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678527 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 04:39:23.686028 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678531 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 04:39:23.686028 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678536 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 04:39:23.686028 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678539 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 04:39:23.686028 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678544 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 04:39:23.686028 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678547 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 04:39:23.686028 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678551 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 04:39:23.686028 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678555 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 04:39:23.686956 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678559 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 04:39:23.686956 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678563 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 04:39:23.686956 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678567 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 04:39:23.686956 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678571 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 04:39:23.686956 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678575 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 23 04:39:23.686956 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678579 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 04:39:23.686956 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678583 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 04:39:23.686956 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678587 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 04:39:23.686956 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678591 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 04:39:23.686956 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678595 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 04:39:23.686956 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678600 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 04:39:23.686956 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678604 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 04:39:23.686956 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678608 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 04:39:23.686956 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678612 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 04:39:23.686956 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678617 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 04:39:23.686956 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678621 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 04:39:23.686956 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678626 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 04:39:23.686956 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678629 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 04:39:23.686956 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678633 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 04:39:23.687575 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678637 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 04:39:23.687575 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678641 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 04:39:23.687575 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678645 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 04:39:23.687575 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678649 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 04:39:23.687575 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678653 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 04:39:23.687575 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678658 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 04:39:23.687575 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678662 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 04:39:23.687575 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678666 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 04:39:23.687575 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678671 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 04:39:23.687575 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678675 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 04:39:23.687575 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678680 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 04:39:23.687575 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678684 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 04:39:23.687575 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678690 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 04:39:23.687575 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678696 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 04:39:23.687575 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678700 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 04:39:23.687575 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678705 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 04:39:23.687575 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678709 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 04:39:23.687575 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678712 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 04:39:23.687575 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678717 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 04:39:23.687575 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678721 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 04:39:23.688274 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678725 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 04:39:23.688274 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678729 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 04:39:23.688274 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678733 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 04:39:23.688274 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678737 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 04:39:23.688274 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678741 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 04:39:23.688274 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678745 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 04:39:23.688274 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678749 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 04:39:23.688274 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678754 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 04:39:23.688274 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678760 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 04:39:23.688274 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678765 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 04:39:23.688274 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678769 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 04:39:23.688274 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678773 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 04:39:23.688274 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678777 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 04:39:23.688274 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678781 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 04:39:23.688274 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678788 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 04:39:23.688274 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678792 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 04:39:23.688274 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678797 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 04:39:23.688274 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678801 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 04:39:23.688274 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678805 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 04:39:23.688274 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678810 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 04:39:23.689053 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678814 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 04:39:23.689053 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678819 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 04:39:23.689053 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678823 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 04:39:23.689053 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678828 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 04:39:23.689053 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678831 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 04:39:23.689053 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:23.678835 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 04:39:23.689053 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.678843 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 04:39:23.689053 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.679006 2562 server.go:962] "Client rotation is on, will bootstrap in background" Jul 23 04:39:23.689053 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.682061 2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 23 04:39:23.689053 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.683829 2562 server.go:1019] "Starting client certificate rotation" Jul 23 04:39:23.689053 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.683942 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 23 04:39:23.689053 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.683985 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 23 04:39:23.689053 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.686809 2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 23 04:39:23.697849 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.697828 2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 23 04:39:23.711995 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.711973 2562 log.go:25] "Validated CRI v1 runtime API" Jul 23 04:39:23.714502 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.714479 2562 log.go:25] "Validated CRI v1 image API" Jul 23 04:39:23.716294 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.716263 2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 23 04:39:23.719466 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.719446 2562 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 bb8f7b8d-1cf3-473a-8d19-d66f095c52a5:/dev/nvme0n1p4 ff72a56c-d7f1-4a37-bf82-e029de6dc156:/dev/nvme0n1p3] Jul 23 04:39:23.719527 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.719467 2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 23 04:39:23.723681 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.723662 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 23 04:39:23.724939 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.724828 2562 manager.go:216] Machine: {Timestamp:2026-07-23 04:39:23.723322625 +0000 UTC m=+0.366556467 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3132342 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ca7d9afba7fa4d6365d796168707c SystemUUID:ec2ca7d9-afba-7fa4-d636-5d796168707c BootID:482fff4d-6011-4a0d-b4cd-da52b2509fec Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:dd:29:90:77:9d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:dd:29:90:77:9d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5a:49:0f:17:49:18 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 23 04:39:23.724939 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.724934 2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 23 04:39:23.725066 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.725017 2562 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 23 04:39:23.728191 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.728166 2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 23 04:39:23.728348 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.728193 2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-96.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 23 04:39:23.728995 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.728984 2562 topology_manager.go:138] "Creating topology manager with none policy" Jul 23 04:39:23.729030 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.728996 2562 container_manager_linux.go:309] "Creating device plugin manager" Jul 23 04:39:23.729030 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.729005 2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 23 04:39:23.729753 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.729742 2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 23 04:39:23.731044 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.731034 2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 23 04:39:23.734123 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.734101 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 23 04:39:23.734296 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.734286 2562 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 23 04:39:23.734332 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.734316 2562 kubelet.go:486] "Attempting to sync node with API server" Jul 23 04:39:23.734363 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.734345 2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 23 04:39:23.734363 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.734360 2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 23 04:39:23.734432 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.734370 2562 kubelet.go:398] "Adding apiserver pod source" Jul 23 04:39:23.734432 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.734405 2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 23 04:39:23.734868 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.734856 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 23 04:39:23.734942 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.734932 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 23 04:39:23.737272 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.737245 2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 23 04:39:23.739494 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.739474 2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 23 04:39:23.739579 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.739516 2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 23 04:39:23.740656 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.740642 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 23 04:39:23.740721 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.740660 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 23 04:39:23.740721 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.740666 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 23 04:39:23.740721 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.740671 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 23 04:39:23.740721 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.740680 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 23 04:39:23.740721 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.740685 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 23 04:39:23.740721 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.740693 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 23 04:39:23.740721 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.740699 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 23 04:39:23.740721 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.740709 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 23 04:39:23.740721 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.740715 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 23 04:39:23.740721 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.740723 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 23 04:39:23.740993 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.740728 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 23 04:39:23.741535 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.741519 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 23 04:39:23.741625 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.741541 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 23 04:39:23.743695 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:23.743665 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 23 04:39:23.743808 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:23.743695 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-96.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 23 04:39:23.744550 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.744510 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-stv4k" Jul 23 04:39:23.746061 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.746048 2562 server.go:1268] "Started kubelet" Jul 23 04:39:23.746191 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.746147 2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 23 04:39:23.746258 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.746199 2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 23 04:39:23.746305 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.746290 2562 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 23 04:39:23.746732 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.746713 2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-96.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 23 04:39:23.746911 ip-10-0-128-96 systemd[1]: Started Kubernetes Kubelet. Jul 23 04:39:23.749533 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.748011 2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 23 04:39:23.749533 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.748305 2562 server.go:310] "Adding debug handlers to kubelet server" Jul 23 04:39:23.750393 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:23.750375 2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 23 04:39:23.752623 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.752608 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-stv4k" Jul 23 04:39:23.754329 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:23.753168 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-96.ec2.internal.18c4d16044bf6c08 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-96.ec2.internal,UID:ip-10-0-128-96.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-96.ec2.internal,},FirstTimestamp:2026-07-23 04:39:23.74602036 +0000 UTC m=+0.389254203,LastTimestamp:2026-07-23 04:39:23.74602036 +0000 UTC m=+0.389254203,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-96.ec2.internal,}" Jul 23 04:39:23.756289 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.756272 2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 23 04:39:23.756373 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.756291 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 23 04:39:23.756488 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:23.756431 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-96.ec2.internal\" not found" Jul 23 04:39:23.756932 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.756914 2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 23 04:39:23.757009 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.756914 2562 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 23 04:39:23.757009 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.756954 2562 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 23 04:39:23.757298 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.757061 2562 reconstruct.go:97] "Volume reconstruction finished" Jul 23 04:39:23.757298 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.757071 2562 reconciler.go:29] "Reconciler: start to sync state" Jul 23 04:39:23.757298 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.757281 2562 factory.go:55] Registering systemd factory Jul 23 04:39:23.757424 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.757316 2562 factory.go:223] Registration of the systemd container factory successfully Jul 23 04:39:23.757625 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.757608 2562 factory.go:153] Registering CRI-O factory Jul 23 04:39:23.757683 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.757628 2562 factory.go:223] Registration of the crio container factory successfully Jul 23 04:39:23.757683 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.757677 2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 23 04:39:23.757772 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.757699 2562 factory.go:103] Registering Raw factory Jul 23 04:39:23.757772 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.757710 2562 manager.go:1196] Started watching for new ooms in manager Jul 23 04:39:23.758079 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.758066 2562 manager.go:349] Starting recovery of all containers Jul 23 04:39:23.758620 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:23.758509 2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-96.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 23 04:39:23.758791 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:23.758622 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 23 04:39:23.769637 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.769620 2562 manager.go:354] Recovery completed Jul 23 04:39:23.775024 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.775011 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 04:39:23.776938 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.776921 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-96.ec2.internal" event="NodeHasSufficientMemory" Jul 23 04:39:23.777002 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.776950 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-96.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 04:39:23.777002 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.776960 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-96.ec2.internal" event="NodeHasSufficientPID" Jul 23 04:39:23.777551 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.777534 2562 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 23 04:39:23.777639 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.777559 2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 23 04:39:23.777639 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.777579 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 23 04:39:23.777707 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.777561 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 04:39:23.778422 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.778404 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-96.ec2.internal" event="NodeHasSufficientMemory" Jul 23 04:39:23.778483 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.778443 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-96.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 04:39:23.778483 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.778461 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-96.ec2.internal" event="NodeHasSufficientPID" Jul 23 04:39:23.778672 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.778660 2562 policy_none.go:49] "None policy: Start" Jul 23 04:39:23.778714 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.778678 2562 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 23 04:39:23.778714 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.778688 2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 23 04:39:23.779640 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.779628 2562 policy_none.go:47] "Start" Jul 23 04:39:23.818693 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.818679 2562 manager.go:337] "Starting Device Plugin manager" Jul 23 04:39:23.818810 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:23.818710 2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 23 04:39:23.818810 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.818719 2562 server.go:87] "Starting device plugin registration server" Jul 23 04:39:23.819009 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.818997 2562 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 23 04:39:23.819044 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.819013 2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 23 04:39:23.819154 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.819132 2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 23 04:39:23.819245 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.819232 2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 23 04:39:23.819245 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.819242 2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 23 04:39:23.819814 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:23.819793 2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 23 04:39:23.819910 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:23.819851 2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-96.ec2.internal\" not found" Jul 23 04:39:23.898940 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.898903 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 23 04:39:23.900285 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.900269 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 23 04:39:23.900371 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.900308 2562 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 23 04:39:23.900371 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.900330 2562 kubelet.go:2448] "Starting kubelet main sync loop" Jul 23 04:39:23.900451 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:23.900369 2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 23 04:39:23.902985 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.902964 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 04:39:23.919547 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.919463 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 04:39:23.920310 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.920290 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-96.ec2.internal" event="NodeHasSufficientMemory" Jul 23 04:39:23.920413 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.920325 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-96.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 04:39:23.920413 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.920337 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-96.ec2.internal" event="NodeHasSufficientPID" Jul 23 04:39:23.920413 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.920361 2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-96.ec2.internal" Jul 23 04:39:23.927160 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:23.927143 2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-96.ec2.internal" Jul 23 04:39:23.927253 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:23.927166 2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-96.ec2.internal\": node \"ip-10-0-128-96.ec2.internal\" not found" Jul 23 04:39:23.945861 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:23.945838 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-96.ec2.internal\" not found" Jul 23 04:39:24.000635 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:24.000598 2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-96.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-96.ec2.internal"] Jul 23 04:39:24.000821 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:24.000697 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 04:39:24.002475 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:24.002461 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-96.ec2.internal" event="NodeHasSufficientMemory" Jul 23 04:39:24.002565 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:24.002495 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-96.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 04:39:24.002565 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:24.002509 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-96.ec2.internal" event="NodeHasSufficientPID" Jul 23 04:39:24.003954 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:24.003937 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 04:39:24.004067 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:24.004051 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-96.ec2.internal" Jul 23 04:39:24.004133 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:24.004085 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 04:39:24.004613 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:24.004599 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-96.ec2.internal" event="NodeHasSufficientMemory" Jul 23 04:39:24.004656 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:24.004626 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-96.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 04:39:24.004656 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:24.004649 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-96.ec2.internal" event="NodeHasSufficientPID" Jul 23 04:39:24.004731 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:24.004697 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-96.ec2.internal" event="NodeHasSufficientMemory" Jul 23 04:39:24.004731 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:24.004727 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-96.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 04:39:24.004788 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:24.004737 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-96.ec2.internal" event="NodeHasSufficientPID" Jul 23 04:39:24.006520 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:24.006503 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-96.ec2.internal" Jul 23 04:39:24.006602 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:24.006533 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 04:39:24.007196 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:24.007183 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-96.ec2.internal" event="NodeHasSufficientMemory" Jul 23 04:39:24.007288 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:24.007210 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-96.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 04:39:24.007288 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:24.007224 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-96.ec2.internal" event="NodeHasSufficientPID" Jul 23 04:39:24.031779 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:24.031756 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-96.ec2.internal\" not found" node="ip-10-0-128-96.ec2.internal" Jul 23 04:39:24.036384 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:24.036367 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-96.ec2.internal\" not found" node="ip-10-0-128-96.ec2.internal" Jul 23 04:39:24.046160 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:24.046134 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-96.ec2.internal\" not found" Jul 23 04:39:24.058457 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:24.058434 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/62fa0ee5fa616af59e13c3ed7949ae7f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-96.ec2.internal\" (UID: \"62fa0ee5fa616af59e13c3ed7949ae7f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-96.ec2.internal" Jul 23 04:39:24.058531 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:24.058458 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/62fa0ee5fa616af59e13c3ed7949ae7f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-96.ec2.internal\" (UID: \"62fa0ee5fa616af59e13c3ed7949ae7f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-96.ec2.internal" Jul 23 04:39:24.058531 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:24.058474 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5666b1587a613009c403213e61aac89c-config\") pod \"kube-apiserver-proxy-ip-10-0-128-96.ec2.internal\" (UID: \"5666b1587a613009c403213e61aac89c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-96.ec2.internal" Jul 23 04:39:24.146668 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:24.146634 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-96.ec2.internal\" not found" Jul 23 04:39:24.159187 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:24.159142 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/62fa0ee5fa616af59e13c3ed7949ae7f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-96.ec2.internal\" (UID: \"62fa0ee5fa616af59e13c3ed7949ae7f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-96.ec2.internal" Jul 23 04:39:24.159187 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:24.159190 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/62fa0ee5fa616af59e13c3ed7949ae7f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-96.ec2.internal\" (UID: \"62fa0ee5fa616af59e13c3ed7949ae7f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-96.ec2.internal" Jul 23 04:39:24.159373 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:24.159209 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5666b1587a613009c403213e61aac89c-config\") pod \"kube-apiserver-proxy-ip-10-0-128-96.ec2.internal\" (UID: \"5666b1587a613009c403213e61aac89c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-96.ec2.internal" Jul 23 04:39:24.159373 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:24.159255 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/62fa0ee5fa616af59e13c3ed7949ae7f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-96.ec2.internal\" (UID: \"62fa0ee5fa616af59e13c3ed7949ae7f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-96.ec2.internal" Jul 23 04:39:24.159373 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:24.159260 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/62fa0ee5fa616af59e13c3ed7949ae7f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-96.ec2.internal\" (UID: \"62fa0ee5fa616af59e13c3ed7949ae7f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-96.ec2.internal" Jul 23 04:39:24.159373 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:24.159309 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5666b1587a613009c403213e61aac89c-config\") pod \"kube-apiserver-proxy-ip-10-0-128-96.ec2.internal\" (UID: \"5666b1587a613009c403213e61aac89c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-96.ec2.internal" Jul 23 04:39:24.247648 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:24.247583 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-96.ec2.internal\" not found" Jul 23 04:39:24.335186 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:24.335161 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-96.ec2.internal" Jul 23 04:39:24.339183 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:24.339164 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-96.ec2.internal" Jul 23 04:39:24.347792 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:24.347643 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-96.ec2.internal\" not found" Jul 23 04:39:25.748956 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:25.748916 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-96.ec2.internal\" not found" Jul 23 04:39:25.849441 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:25.849410 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-96.ec2.internal\" not found" Jul 23 04:39:25.950128 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:25.950085 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-96.ec2.internal\" not found" Jul 23 04:39:25.983606 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:25.983573 2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 23 04:39:25.984017 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:25.983734 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 23 04:39:26.050791 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:26.050762 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-96.ec2.internal\" not found" Jul 23 04:39:26.057998 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:26.057960 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-22 04:34:25 +0000 UTC" deadline="2028-04-19 17:21:31.13785595 +0000 UTC" Jul 23 04:39:26.058067 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:26.057999 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15276h42m5.079859747s" Jul 23 04:39:26.151538 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:26.151496 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-96.ec2.internal\" not found" Jul 23 04:39:26.228238 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:26.228206 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5666b1587a613009c403213e61aac89c.slice/crio-1ade9c247fff30945033a9ebe362e8daf890ef89d3737fa9cd44e506e4fda96b WatchSource:0}: Error finding container 1ade9c247fff30945033a9ebe362e8daf890ef89d3737fa9cd44e506e4fda96b: Status 404 returned error can't find the container with id 1ade9c247fff30945033a9ebe362e8daf890ef89d3737fa9cd44e506e4fda96b Jul 23 04:39:26.228597 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:26.228579 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62fa0ee5fa616af59e13c3ed7949ae7f.slice/crio-87e5c701ab7c63c489ee77688e4a4034246b728455adfda8238a1c75186d55f8 WatchSource:0}: Error finding container 87e5c701ab7c63c489ee77688e4a4034246b728455adfda8238a1c75186d55f8: Status 404 returned error can't find the container with id 87e5c701ab7c63c489ee77688e4a4034246b728455adfda8238a1c75186d55f8 Jul 23 04:39:26.233458 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:26.233418 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 04:39:26.252383 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:26.252347 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-96.ec2.internal\" not found" Jul 23 04:39:26.352986 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:26.352948 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-96.ec2.internal\" not found" Jul 23 04:39:26.382758 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:26.382686 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 04:39:26.403929 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:26.403903 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 04:39:26.457987 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:26.457960 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-96.ec2.internal" Jul 23 04:39:26.470575 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:26.470547 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 23 04:39:26.473143 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:26.473125 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-96.ec2.internal" Jul 23 04:39:26.484637 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:26.484617 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 23 04:39:26.589920 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:26.588313 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 04:39:27.036560 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.036525 2562 apiserver.go:52] "Watching apiserver" Jul 23 04:39:27.044849 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.044810 2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 23 04:39:27.045252 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.045222 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-mpnqv","openshift-network-diagnostics/network-check-target-7dsv7","openshift-network-operator/iptables-alerter-htlgx","kube-system/konnectivity-agent-zhq6d","kube-system/kube-apiserver-proxy-ip-10-0-128-96.ec2.internal","openshift-cluster-node-tuning-operator/tuned-q2kph","openshift-image-registry/node-ca-grlbc","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-96.ec2.internal","openshift-ovn-kubernetes/ovnkube-node-vphbq","kube-system/global-pull-secret-syncer-cbzpv","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9nzz9","openshift-dns/node-resolver-57bmk","openshift-multus/multus-additional-cni-plugins-zzgwf","openshift-multus/multus-w7d9f"] Jul 23 04:39:27.047315 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.047290 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-grlbc" Jul 23 04:39:27.049763 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.049281 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9nzz9" Jul 23 04:39:27.050135 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.050115 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 23 04:39:27.050344 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.050329 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 23 04:39:27.050567 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.050550 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4vhtm\"" Jul 23 04:39:27.050641 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.050630 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 23 04:39:27.050891 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.050873 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-htlgx" Jul 23 04:39:27.052735 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.051681 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 23 04:39:27.052735 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.051911 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 23 04:39:27.052735 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.052076 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 23 04:39:27.052735 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.052235 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-fvr45\"" Jul 23 04:39:27.053123 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.053103 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 23 04:39:27.053371 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.053354 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 23 04:39:27.053553 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.053533 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 23 04:39:27.053745 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.053730 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-cxb56\"" Jul 23 04:39:27.053849 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.053816 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zhq6d" Jul 23 04:39:27.055096 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.055075 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.056026 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.056009 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9mvsc\"" Jul 23 04:39:27.056158 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.056138 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 23 04:39:27.056247 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.056144 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 23 04:39:27.056541 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.056523 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.057242 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.057222 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 23 04:39:27.057242 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.057255 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 23 04:39:27.059381 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.058568 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 23 04:39:27.059381 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.058615 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.059381 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.058870 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 23 04:39:27.059381 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.059210 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 23 04:39:27.059825 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.059611 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 23 04:39:27.059825 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.059777 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4w578\"" Jul 23 04:39:27.059825 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.059799 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 23 04:39:27.060088 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.060062 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-c2wvc\"" Jul 23 04:39:27.061007 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.060713 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 23 04:39:27.061251 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.061238 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-z9sns\"" Jul 23 04:39:27.061763 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.061743 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 23 04:39:27.061928 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.061906 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zzgwf" Jul 23 04:39:27.062545 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.062013 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 23 04:39:27.062545 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.062025 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 23 04:39:27.062545 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.062332 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 23 04:39:27.062545 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.062402 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 23 04:39:27.063056 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.063038 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dsv7" Jul 23 04:39:27.063139 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:27.063112 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dsv7" podUID="6346ee84-530c-41f1-9f8a-6d83f79daa5e" Jul 23 04:39:27.069024 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.069002 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 23 04:39:27.069672 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.069655 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-f8gxv\"" Jul 23 04:39:27.071908 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.070104 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 23 04:39:27.071908 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.070399 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-os-release\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.071908 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.070423 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c49b7ef0-9710-41cd-9d89-145654e92fc6-cnibin\") pod \"multus-additional-cni-plugins-zzgwf\" (UID: \"c49b7ef0-9710-41cd-9d89-145654e92fc6\") " pod="openshift-multus/multus-additional-cni-plugins-zzgwf" Jul 23 04:39:27.071908 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.070443 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-run-ovn\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.071908 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.070462 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-multus-socket-dir-parent\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.071908 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.070482 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c49b7ef0-9710-41cd-9d89-145654e92fc6-os-release\") pod \"multus-additional-cni-plugins-zzgwf\" (UID: \"c49b7ef0-9710-41cd-9d89-145654e92fc6\") " pod="openshift-multus/multus-additional-cni-plugins-zzgwf" Jul 23 04:39:27.071908 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.070503 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c49b7ef0-9710-41cd-9d89-145654e92fc6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zzgwf\" (UID: \"c49b7ef0-9710-41cd-9d89-145654e92fc6\") " pod="openshift-multus/multus-additional-cni-plugins-zzgwf" Jul 23 04:39:27.071908 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.070537 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ba17063f-a4cc-4378-afe1-6b4aad36adef-konnectivity-ca\") pod \"konnectivity-agent-zhq6d\" (UID: \"ba17063f-a4cc-4378-afe1-6b4aad36adef\") " pod="kube-system/konnectivity-agent-zhq6d" Jul 23 04:39:27.071908 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.070553 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/43514e24-f4a7-4dfc-b45d-12d28e0a4bfc-sys-fs\") pod \"aws-ebs-csi-driver-node-9nzz9\" (UID: \"43514e24-f4a7-4dfc-b45d-12d28e0a4bfc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9nzz9" Jul 23 04:39:27.071908 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.070574 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c49b7ef0-9710-41cd-9d89-145654e92fc6-system-cni-dir\") pod \"multus-additional-cni-plugins-zzgwf\" (UID: \"c49b7ef0-9710-41cd-9d89-145654e92fc6\") " pod="openshift-multus/multus-additional-cni-plugins-zzgwf" Jul 23 04:39:27.071908 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.070592 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-run-systemd\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.071908 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.070610 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-var-lib-openvswitch\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.071908 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.070628 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-host-var-lib-cni-multus\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.071908 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.070645 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-etc-kubernetes\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.071908 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.070662 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b3007292-2534-4dc0-82a3-41d6a7d34c0c-env-overrides\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.071908 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.070680 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-etc-sysconfig\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.071908 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.070696 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-host\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.073133 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.070716 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/67a7fb5e-8e74-4275-8abd-56f340394bd0-host-slash\") pod \"iptables-alerter-htlgx\" (UID: \"67a7fb5e-8e74-4275-8abd-56f340394bd0\") " pod="openshift-network-operator/iptables-alerter-htlgx" Jul 23 04:39:27.073133 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.070734 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-cnibin\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.073133 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.070748 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-host-run-k8s-cni-cncf-io\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.073133 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.070766 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-etc-openvswitch\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.073133 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.070784 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/67a7fb5e-8e74-4275-8abd-56f340394bd0-iptables-alerter-script\") pod \"iptables-alerter-htlgx\" (UID: \"67a7fb5e-8e74-4275-8abd-56f340394bd0\") " pod="openshift-network-operator/iptables-alerter-htlgx" Jul 23 04:39:27.073133 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.070802 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-system-cni-dir\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.073133 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.070818 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-host-run-netns\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.073133 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.070872 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-hostroot\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.073133 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.070893 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-var-lib-kubelet\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.073133 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.070913 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-host-slash\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.073133 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.070932 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-run-openvswitch\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.073133 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.070947 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/43514e24-f4a7-4dfc-b45d-12d28e0a4bfc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9nzz9\" (UID: \"43514e24-f4a7-4dfc-b45d-12d28e0a4bfc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9nzz9" Jul 23 04:39:27.073133 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.070992 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/43514e24-f4a7-4dfc-b45d-12d28e0a4bfc-device-dir\") pod \"aws-ebs-csi-driver-node-9nzz9\" (UID: \"43514e24-f4a7-4dfc-b45d-12d28e0a4bfc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9nzz9" Jul 23 04:39:27.073133 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.071013 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-run\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.073133 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.071032 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b3007292-2534-4dc0-82a3-41d6a7d34c0c-ovnkube-script-lib\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.073133 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.071076 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-multus-cni-dir\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.073133 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.071209 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-etc-kubernetes\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.074005 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.071249 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-host-cni-bin\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.074005 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.071288 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c49b7ef0-9710-41cd-9d89-145654e92fc6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zzgwf\" (UID: \"c49b7ef0-9710-41cd-9d89-145654e92fc6\") " pod="openshift-multus/multus-additional-cni-plugins-zzgwf" Jul 23 04:39:27.074005 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.071321 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-etc-modprobe-d\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.074005 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.071351 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-etc-systemd\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.074005 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.071382 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b3007292-2534-4dc0-82a3-41d6a7d34c0c-ovn-node-metrics-cert\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.074005 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.071533 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tk595\" (UniqueName: \"kubernetes.io/projected/b3007292-2534-4dc0-82a3-41d6a7d34c0c-kube-api-access-tk595\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.074005 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.071573 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/43514e24-f4a7-4dfc-b45d-12d28e0a4bfc-socket-dir\") pod \"aws-ebs-csi-driver-node-9nzz9\" (UID: \"43514e24-f4a7-4dfc-b45d-12d28e0a4bfc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9nzz9" Jul 23 04:39:27.074005 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.071602 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7szxj\" (UniqueName: \"kubernetes.io/projected/43514e24-f4a7-4dfc-b45d-12d28e0a4bfc-kube-api-access-7szxj\") pod \"aws-ebs-csi-driver-node-9nzz9\" (UID: \"43514e24-f4a7-4dfc-b45d-12d28e0a4bfc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9nzz9" Jul 23 04:39:27.074005 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.071642 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c49b7ef0-9710-41cd-9d89-145654e92fc6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zzgwf\" (UID: \"c49b7ef0-9710-41cd-9d89-145654e92fc6\") " pod="openshift-multus/multus-additional-cni-plugins-zzgwf" Jul 23 04:39:27.074005 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.071698 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-node-log\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.074005 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.071788 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d69de93b-d93f-4f10-b618-0cdcb545197a-host\") pod \"node-ca-grlbc\" (UID: \"d69de93b-d93f-4f10-b618-0cdcb545197a\") " pod="openshift-image-registry/node-ca-grlbc" Jul 23 04:39:27.074005 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.071858 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svpl8\" (UniqueName: \"kubernetes.io/projected/67a7fb5e-8e74-4275-8abd-56f340394bd0-kube-api-access-svpl8\") pod \"iptables-alerter-htlgx\" (UID: \"67a7fb5e-8e74-4275-8abd-56f340394bd0\") " pod="openshift-network-operator/iptables-alerter-htlgx" Jul 23 04:39:27.074005 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.071900 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-multus-conf-dir\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.074005 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.071945 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mbkq\" (UniqueName: \"kubernetes.io/projected/0bf5e605-388c-4f50-8716-c350190d11d9-kube-api-access-8mbkq\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.074005 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.072027 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-etc-sysctl-conf\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.074005 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.072102 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-sys\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.074782 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.072155 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-tmp\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.074782 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.072209 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b3007292-2534-4dc0-82a3-41d6a7d34c0c-ovnkube-config\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.074782 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.072250 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d69de93b-d93f-4f10-b618-0cdcb545197a-serviceca\") pod \"node-ca-grlbc\" (UID: \"d69de93b-d93f-4f10-b618-0cdcb545197a\") " pod="openshift-image-registry/node-ca-grlbc" Jul 23 04:39:27.074782 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.072276 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-lib-modules\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.074782 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.072315 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-host-cni-netd\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.074782 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.072419 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75z5n\" (UniqueName: \"kubernetes.io/projected/d69de93b-d93f-4f10-b618-0cdcb545197a-kube-api-access-75z5n\") pod \"node-ca-grlbc\" (UID: \"d69de93b-d93f-4f10-b618-0cdcb545197a\") " pod="openshift-image-registry/node-ca-grlbc" Jul 23 04:39:27.074782 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.072454 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/43514e24-f4a7-4dfc-b45d-12d28e0a4bfc-registration-dir\") pod \"aws-ebs-csi-driver-node-9nzz9\" (UID: \"43514e24-f4a7-4dfc-b45d-12d28e0a4bfc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9nzz9" Jul 23 04:39:27.074782 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.072499 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-host-var-lib-kubelet\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.074782 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.072529 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-host-run-netns\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.074782 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.072584 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0bf5e605-388c-4f50-8716-c350190d11d9-cni-binary-copy\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.074782 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.072665 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-host-var-lib-cni-bin\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.074782 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.072700 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c49b7ef0-9710-41cd-9d89-145654e92fc6-cni-binary-copy\") pod \"multus-additional-cni-plugins-zzgwf\" (UID: \"c49b7ef0-9710-41cd-9d89-145654e92fc6\") " pod="openshift-multus/multus-additional-cni-plugins-zzgwf" Jul 23 04:39:27.074782 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.074021 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmx8b\" (UniqueName: \"kubernetes.io/projected/c49b7ef0-9710-41cd-9d89-145654e92fc6-kube-api-access-wmx8b\") pod \"multus-additional-cni-plugins-zzgwf\" (UID: \"c49b7ef0-9710-41cd-9d89-145654e92fc6\") " pod="openshift-multus/multus-additional-cni-plugins-zzgwf" Jul 23 04:39:27.074782 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.074064 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-etc-sysctl-d\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.074782 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.074517 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x5h7\" (UniqueName: \"kubernetes.io/projected/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-kube-api-access-4x5h7\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.074782 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.074540 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-host-kubelet\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.075574 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.074560 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-host-run-ovn-kubernetes\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.075574 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.074581 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0bf5e605-388c-4f50-8716-c350190d11d9-multus-daemon-config\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.075574 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.074602 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-host-run-multus-certs\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.075574 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.074617 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ba17063f-a4cc-4378-afe1-6b4aad36adef-agent-certs\") pod \"konnectivity-agent-zhq6d\" (UID: \"ba17063f-a4cc-4378-afe1-6b4aad36adef\") " pod="kube-system/konnectivity-agent-zhq6d" Jul 23 04:39:27.075574 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.074486 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbzpv" Jul 23 04:39:27.075574 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.074654 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-etc-tuned\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.075574 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:27.074679 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbzpv" podUID="555665cd-6364-4d9f-8326-4973b140cc64" Jul 23 04:39:27.075574 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.074682 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-systemd-units\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.075574 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.074705 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-log-socket\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.075574 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.074736 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.075574 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.074757 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/43514e24-f4a7-4dfc-b45d-12d28e0a4bfc-etc-selinux\") pod \"aws-ebs-csi-driver-node-9nzz9\" (UID: \"43514e24-f4a7-4dfc-b45d-12d28e0a4bfc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9nzz9" Jul 23 04:39:27.075574 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.075063 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mpnqv" Jul 23 04:39:27.075574 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:27.075123 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mpnqv" podUID="dab60874-8b3b-411d-b58e-4ec857bc9629" Jul 23 04:39:27.076323 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.076301 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 23 04:39:27.076993 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.076975 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-57bmk" Jul 23 04:39:27.079352 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.079208 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 23 04:39:27.079352 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.079340 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hcpg6\"" Jul 23 04:39:27.079489 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.079461 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 23 04:39:27.099676 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.099654 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-dw8zx" Jul 23 04:39:27.105189 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.105168 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-dw8zx" Jul 23 04:39:27.158902 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.158872 2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 23 04:39:27.175380 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.175339 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czjdx\" (UniqueName: \"kubernetes.io/projected/6346ee84-530c-41f1-9f8a-6d83f79daa5e-kube-api-access-czjdx\") pod \"network-check-target-7dsv7\" (UID: \"6346ee84-530c-41f1-9f8a-6d83f79daa5e\") " pod="openshift-network-diagnostics/network-check-target-7dsv7" Jul 23 04:39:27.175556 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.175392 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-run\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.175556 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.175452 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b3007292-2534-4dc0-82a3-41d6a7d34c0c-ovnkube-script-lib\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.175556 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.175490 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-multus-cni-dir\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.175556 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.175518 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-etc-kubernetes\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.175556 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.175539 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-run\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.175556 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.175544 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-host-cni-bin\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.175854 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.175572 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c49b7ef0-9710-41cd-9d89-145654e92fc6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zzgwf\" (UID: \"c49b7ef0-9710-41cd-9d89-145654e92fc6\") " pod="openshift-multus/multus-additional-cni-plugins-zzgwf" Jul 23 04:39:27.175854 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.175601 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-etc-kubernetes\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.175854 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.175611 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-host-cni-bin\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.175854 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.175703 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-multus-cni-dir\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.175854 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.175706 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9a49f0e-338a-47ac-90f7-281dae5538e4-tmp-dir\") pod \"node-resolver-57bmk\" (UID: \"b9a49f0e-338a-47ac-90f7-281dae5538e4\") " pod="openshift-dns/node-resolver-57bmk" Jul 23 04:39:27.175854 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.175751 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c49b7ef0-9710-41cd-9d89-145654e92fc6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zzgwf\" (UID: \"c49b7ef0-9710-41cd-9d89-145654e92fc6\") " pod="openshift-multus/multus-additional-cni-plugins-zzgwf" Jul 23 04:39:27.175854 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.175757 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-etc-modprobe-d\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.175854 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.175784 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-etc-systemd\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.175854 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.175810 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b3007292-2534-4dc0-82a3-41d6a7d34c0c-ovn-node-metrics-cert\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.176267 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.175858 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tk595\" (UniqueName: \"kubernetes.io/projected/b3007292-2534-4dc0-82a3-41d6a7d34c0c-kube-api-access-tk595\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.176267 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.175869 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-etc-systemd\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.176267 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.175858 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-etc-modprobe-d\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.176267 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.175966 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/43514e24-f4a7-4dfc-b45d-12d28e0a4bfc-socket-dir\") pod \"aws-ebs-csi-driver-node-9nzz9\" (UID: \"43514e24-f4a7-4dfc-b45d-12d28e0a4bfc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9nzz9" Jul 23 04:39:27.176267 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.175986 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7szxj\" (UniqueName: \"kubernetes.io/projected/43514e24-f4a7-4dfc-b45d-12d28e0a4bfc-kube-api-access-7szxj\") pod \"aws-ebs-csi-driver-node-9nzz9\" (UID: \"43514e24-f4a7-4dfc-b45d-12d28e0a4bfc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9nzz9" Jul 23 04:39:27.176267 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.176010 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c49b7ef0-9710-41cd-9d89-145654e92fc6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zzgwf\" (UID: \"c49b7ef0-9710-41cd-9d89-145654e92fc6\") " pod="openshift-multus/multus-additional-cni-plugins-zzgwf" Jul 23 04:39:27.176267 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.176029 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-node-log\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.176267 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.176043 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d69de93b-d93f-4f10-b618-0cdcb545197a-host\") pod \"node-ca-grlbc\" (UID: \"d69de93b-d93f-4f10-b618-0cdcb545197a\") " pod="openshift-image-registry/node-ca-grlbc" Jul 23 04:39:27.176267 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.176057 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svpl8\" (UniqueName: \"kubernetes.io/projected/67a7fb5e-8e74-4275-8abd-56f340394bd0-kube-api-access-svpl8\") pod \"iptables-alerter-htlgx\" (UID: \"67a7fb5e-8e74-4275-8abd-56f340394bd0\") " pod="openshift-network-operator/iptables-alerter-htlgx" Jul 23 04:39:27.176267 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.176077 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-multus-conf-dir\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.176267 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.176098 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mbkq\" (UniqueName: \"kubernetes.io/projected/0bf5e605-388c-4f50-8716-c350190d11d9-kube-api-access-8mbkq\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.176267 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.176125 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b3007292-2534-4dc0-82a3-41d6a7d34c0c-ovnkube-script-lib\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.176267 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.176131 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-etc-sysctl-conf\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.176267 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.176130 2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 23 04:39:27.176267 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.176147 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-sys\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.176267 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.176146 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-node-log\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.176267 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.176166 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-tmp\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.176267 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.176181 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-multus-conf-dir\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.177147 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.176186 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b3007292-2534-4dc0-82a3-41d6a7d34c0c-ovnkube-config\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.177147 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.176200 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d69de93b-d93f-4f10-b618-0cdcb545197a-serviceca\") pod \"node-ca-grlbc\" (UID: \"d69de93b-d93f-4f10-b618-0cdcb545197a\") " pod="openshift-image-registry/node-ca-grlbc" Jul 23 04:39:27.177147 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.176207 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d69de93b-d93f-4f10-b618-0cdcb545197a-host\") pod \"node-ca-grlbc\" (UID: \"d69de93b-d93f-4f10-b618-0cdcb545197a\") " pod="openshift-image-registry/node-ca-grlbc" Jul 23 04:39:27.177147 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.176223 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-lib-modules\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.177147 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.176255 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/43514e24-f4a7-4dfc-b45d-12d28e0a4bfc-socket-dir\") pod \"aws-ebs-csi-driver-node-9nzz9\" (UID: \"43514e24-f4a7-4dfc-b45d-12d28e0a4bfc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9nzz9" Jul 23 04:39:27.177147 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.176297 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-lib-modules\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.177147 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.176397 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-host-cni-netd\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.177147 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.176425 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75z5n\" (UniqueName: \"kubernetes.io/projected/d69de93b-d93f-4f10-b618-0cdcb545197a-kube-api-access-75z5n\") pod \"node-ca-grlbc\" (UID: \"d69de93b-d93f-4f10-b618-0cdcb545197a\") " pod="openshift-image-registry/node-ca-grlbc" Jul 23 04:39:27.177147 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.176446 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/43514e24-f4a7-4dfc-b45d-12d28e0a4bfc-registration-dir\") pod \"aws-ebs-csi-driver-node-9nzz9\" (UID: \"43514e24-f4a7-4dfc-b45d-12d28e0a4bfc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9nzz9" Jul 23 04:39:27.177147 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.176471 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-host-var-lib-kubelet\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.177147 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.176492 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-host-run-netns\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.177147 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.176491 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-etc-sysctl-conf\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.177147 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.176516 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-host-cni-netd\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.177147 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.176524 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-sys\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.177147 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.176536 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0bf5e605-388c-4f50-8716-c350190d11d9-cni-binary-copy\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.177147 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.176547 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/43514e24-f4a7-4dfc-b45d-12d28e0a4bfc-registration-dir\") pod \"aws-ebs-csi-driver-node-9nzz9\" (UID: \"43514e24-f4a7-4dfc-b45d-12d28e0a4bfc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9nzz9" Jul 23 04:39:27.177147 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.176576 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-host-var-lib-cni-bin\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.177938 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.176678 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c49b7ef0-9710-41cd-9d89-145654e92fc6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zzgwf\" (UID: \"c49b7ef0-9710-41cd-9d89-145654e92fc6\") " pod="openshift-multus/multus-additional-cni-plugins-zzgwf" Jul 23 04:39:27.177938 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.176951 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b3007292-2534-4dc0-82a3-41d6a7d34c0c-ovnkube-config\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.177938 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.176615 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-host-var-lib-kubelet\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.177938 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.176616 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c49b7ef0-9710-41cd-9d89-145654e92fc6-cni-binary-copy\") pod \"multus-additional-cni-plugins-zzgwf\" (UID: \"c49b7ef0-9710-41cd-9d89-145654e92fc6\") " pod="openshift-multus/multus-additional-cni-plugins-zzgwf" Jul 23 04:39:27.177938 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.177101 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmx8b\" (UniqueName: \"kubernetes.io/projected/c49b7ef0-9710-41cd-9d89-145654e92fc6-kube-api-access-wmx8b\") pod \"multus-additional-cni-plugins-zzgwf\" (UID: \"c49b7ef0-9710-41cd-9d89-145654e92fc6\") " pod="openshift-multus/multus-additional-cni-plugins-zzgwf" Jul 23 04:39:27.177938 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.177128 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-etc-sysctl-d\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.177938 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.177151 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x5h7\" (UniqueName: \"kubernetes.io/projected/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-kube-api-access-4x5h7\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.177938 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.177191 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-host-kubelet\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.177938 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.177217 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-host-run-ovn-kubernetes\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.177938 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.177240 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0bf5e605-388c-4f50-8716-c350190d11d9-multus-daemon-config\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.177938 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.177264 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-host-run-multus-certs\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.177938 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.177291 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dab60874-8b3b-411d-b58e-4ec857bc9629-metrics-certs\") pod \"network-metrics-daemon-mpnqv\" (UID: \"dab60874-8b3b-411d-b58e-4ec857bc9629\") " pod="openshift-multus/network-metrics-daemon-mpnqv" Jul 23 04:39:27.177938 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.177315 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vchx\" (UniqueName: \"kubernetes.io/projected/b9a49f0e-338a-47ac-90f7-281dae5538e4-kube-api-access-2vchx\") pod \"node-resolver-57bmk\" (UID: \"b9a49f0e-338a-47ac-90f7-281dae5538e4\") " pod="openshift-dns/node-resolver-57bmk" Jul 23 04:39:27.177938 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.177343 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ba17063f-a4cc-4378-afe1-6b4aad36adef-agent-certs\") pod \"konnectivity-agent-zhq6d\" (UID: \"ba17063f-a4cc-4378-afe1-6b4aad36adef\") " pod="kube-system/konnectivity-agent-zhq6d" Jul 23 04:39:27.177938 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.177345 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d69de93b-d93f-4f10-b618-0cdcb545197a-serviceca\") pod \"node-ca-grlbc\" (UID: \"d69de93b-d93f-4f10-b618-0cdcb545197a\") " pod="openshift-image-registry/node-ca-grlbc" Jul 23 04:39:27.177938 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.177368 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-etc-tuned\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.177938 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.177295 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-host-run-netns\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.178743 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.177412 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-host-var-lib-cni-bin\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.178743 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.177428 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-systemd-units\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.178743 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.177469 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-host-run-ovn-kubernetes\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.178743 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.177515 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0bf5e605-388c-4f50-8716-c350190d11d9-cni-binary-copy\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.178743 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.177544 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-host-kubelet\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.178743 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.177393 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-systemd-units\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.178743 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.177604 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-log-socket\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.178743 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.177640 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.178743 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.177668 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/43514e24-f4a7-4dfc-b45d-12d28e0a4bfc-etc-selinux\") pod \"aws-ebs-csi-driver-node-9nzz9\" (UID: \"43514e24-f4a7-4dfc-b45d-12d28e0a4bfc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9nzz9" Jul 23 04:39:27.178743 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.177718 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-os-release\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.178743 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.177740 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c49b7ef0-9710-41cd-9d89-145654e92fc6-cnibin\") pod \"multus-additional-cni-plugins-zzgwf\" (UID: \"c49b7ef0-9710-41cd-9d89-145654e92fc6\") " pod="openshift-multus/multus-additional-cni-plugins-zzgwf" Jul 23 04:39:27.178743 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.177764 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-run-ovn\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.178743 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.177788 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-multus-socket-dir-parent\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.178743 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.177811 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c49b7ef0-9710-41cd-9d89-145654e92fc6-os-release\") pod \"multus-additional-cni-plugins-zzgwf\" (UID: \"c49b7ef0-9710-41cd-9d89-145654e92fc6\") " pod="openshift-multus/multus-additional-cni-plugins-zzgwf" Jul 23 04:39:27.178743 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.177889 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c49b7ef0-9710-41cd-9d89-145654e92fc6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zzgwf\" (UID: \"c49b7ef0-9710-41cd-9d89-145654e92fc6\") " pod="openshift-multus/multus-additional-cni-plugins-zzgwf" Jul 23 04:39:27.178743 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.177925 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/555665cd-6364-4d9f-8326-4973b140cc64-original-pull-secret\") pod \"global-pull-secret-syncer-cbzpv\" (UID: \"555665cd-6364-4d9f-8326-4973b140cc64\") " pod="kube-system/global-pull-secret-syncer-cbzpv" Jul 23 04:39:27.178743 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.177951 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ba17063f-a4cc-4378-afe1-6b4aad36adef-konnectivity-ca\") pod \"konnectivity-agent-zhq6d\" (UID: \"ba17063f-a4cc-4378-afe1-6b4aad36adef\") " pod="kube-system/konnectivity-agent-zhq6d" Jul 23 04:39:27.179587 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.177979 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/43514e24-f4a7-4dfc-b45d-12d28e0a4bfc-sys-fs\") pod \"aws-ebs-csi-driver-node-9nzz9\" (UID: \"43514e24-f4a7-4dfc-b45d-12d28e0a4bfc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9nzz9" Jul 23 04:39:27.179587 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178007 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c49b7ef0-9710-41cd-9d89-145654e92fc6-system-cni-dir\") pod \"multus-additional-cni-plugins-zzgwf\" (UID: \"c49b7ef0-9710-41cd-9d89-145654e92fc6\") " pod="openshift-multus/multus-additional-cni-plugins-zzgwf" Jul 23 04:39:27.179587 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178030 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b9a49f0e-338a-47ac-90f7-281dae5538e4-hosts-file\") pod \"node-resolver-57bmk\" (UID: \"b9a49f0e-338a-47ac-90f7-281dae5538e4\") " pod="openshift-dns/node-resolver-57bmk" Jul 23 04:39:27.179587 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178056 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-run-systemd\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.179587 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178072 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c49b7ef0-9710-41cd-9d89-145654e92fc6-cni-binary-copy\") pod \"multus-additional-cni-plugins-zzgwf\" (UID: \"c49b7ef0-9710-41cd-9d89-145654e92fc6\") " pod="openshift-multus/multus-additional-cni-plugins-zzgwf" Jul 23 04:39:27.179587 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178084 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-var-lib-openvswitch\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.179587 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178092 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-run-ovn\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.179587 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178111 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-host-var-lib-cni-multus\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.179587 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178114 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-etc-sysctl-d\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.179587 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178056 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0bf5e605-388c-4f50-8716-c350190d11d9-multus-daemon-config\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.179587 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178167 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-var-lib-openvswitch\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.179587 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178179 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/43514e24-f4a7-4dfc-b45d-12d28e0a4bfc-sys-fs\") pod \"aws-ebs-csi-driver-node-9nzz9\" (UID: \"43514e24-f4a7-4dfc-b45d-12d28e0a4bfc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9nzz9" Jul 23 04:39:27.179587 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178207 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-log-socket\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.179587 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178214 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.179587 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178259 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-os-release\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.179587 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178260 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c49b7ef0-9710-41cd-9d89-145654e92fc6-os-release\") pod \"multus-additional-cni-plugins-zzgwf\" (UID: \"c49b7ef0-9710-41cd-9d89-145654e92fc6\") " pod="openshift-multus/multus-additional-cni-plugins-zzgwf" Jul 23 04:39:27.179587 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178292 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c49b7ef0-9710-41cd-9d89-145654e92fc6-cnibin\") pod \"multus-additional-cni-plugins-zzgwf\" (UID: \"c49b7ef0-9710-41cd-9d89-145654e92fc6\") " pod="openshift-multus/multus-additional-cni-plugins-zzgwf" Jul 23 04:39:27.180371 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178295 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-host-run-multus-certs\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.180371 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178325 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-host-var-lib-cni-multus\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.180371 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178363 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-run-systemd\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.180371 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178367 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/43514e24-f4a7-4dfc-b45d-12d28e0a4bfc-etc-selinux\") pod \"aws-ebs-csi-driver-node-9nzz9\" (UID: \"43514e24-f4a7-4dfc-b45d-12d28e0a4bfc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9nzz9" Jul 23 04:39:27.180371 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178403 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-multus-socket-dir-parent\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.180371 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178411 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c49b7ef0-9710-41cd-9d89-145654e92fc6-system-cni-dir\") pod \"multus-additional-cni-plugins-zzgwf\" (UID: \"c49b7ef0-9710-41cd-9d89-145654e92fc6\") " pod="openshift-multus/multus-additional-cni-plugins-zzgwf" Jul 23 04:39:27.180371 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178460 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/555665cd-6364-4d9f-8326-4973b140cc64-dbus\") pod \"global-pull-secret-syncer-cbzpv\" (UID: \"555665cd-6364-4d9f-8326-4973b140cc64\") " pod="kube-system/global-pull-secret-syncer-cbzpv" Jul 23 04:39:27.180371 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178492 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-etc-kubernetes\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.180371 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178520 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b3007292-2534-4dc0-82a3-41d6a7d34c0c-env-overrides\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.180371 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178549 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-etc-kubernetes\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.180371 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178575 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c49b7ef0-9710-41cd-9d89-145654e92fc6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zzgwf\" (UID: \"c49b7ef0-9710-41cd-9d89-145654e92fc6\") " pod="openshift-multus/multus-additional-cni-plugins-zzgwf" Jul 23 04:39:27.180371 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178606 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-etc-sysconfig\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.180371 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178627 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ba17063f-a4cc-4378-afe1-6b4aad36adef-konnectivity-ca\") pod \"konnectivity-agent-zhq6d\" (UID: \"ba17063f-a4cc-4378-afe1-6b4aad36adef\") " pod="kube-system/konnectivity-agent-zhq6d" Jul 23 04:39:27.180371 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178644 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-host\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.180371 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178648 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-etc-sysconfig\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.180371 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178675 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/67a7fb5e-8e74-4275-8abd-56f340394bd0-host-slash\") pod \"iptables-alerter-htlgx\" (UID: \"67a7fb5e-8e74-4275-8abd-56f340394bd0\") " pod="openshift-network-operator/iptables-alerter-htlgx" Jul 23 04:39:27.180371 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178690 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-host\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.180371 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178725 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-cnibin\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.181138 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178752 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-host-run-k8s-cni-cncf-io\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.181138 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178800 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/67a7fb5e-8e74-4275-8abd-56f340394bd0-host-slash\") pod \"iptables-alerter-htlgx\" (UID: \"67a7fb5e-8e74-4275-8abd-56f340394bd0\") " pod="openshift-network-operator/iptables-alerter-htlgx" Jul 23 04:39:27.181138 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178813 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-cnibin\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.181138 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178823 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/555665cd-6364-4d9f-8326-4973b140cc64-kubelet-config\") pod \"global-pull-secret-syncer-cbzpv\" (UID: \"555665cd-6364-4d9f-8326-4973b140cc64\") " pod="kube-system/global-pull-secret-syncer-cbzpv" Jul 23 04:39:27.181138 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178880 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-host-run-k8s-cni-cncf-io\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.181138 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178923 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-etc-openvswitch\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.181138 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178925 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b3007292-2534-4dc0-82a3-41d6a7d34c0c-env-overrides\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.181138 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.178952 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-etc-openvswitch\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.181138 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.179001 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/67a7fb5e-8e74-4275-8abd-56f340394bd0-iptables-alerter-script\") pod \"iptables-alerter-htlgx\" (UID: \"67a7fb5e-8e74-4275-8abd-56f340394bd0\") " pod="openshift-network-operator/iptables-alerter-htlgx" Jul 23 04:39:27.181138 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.179106 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-system-cni-dir\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.181138 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.179131 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-host-run-netns\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.181138 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.179154 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-hostroot\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.181138 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.179178 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xncd8\" (UniqueName: \"kubernetes.io/projected/dab60874-8b3b-411d-b58e-4ec857bc9629-kube-api-access-xncd8\") pod \"network-metrics-daemon-mpnqv\" (UID: \"dab60874-8b3b-411d-b58e-4ec857bc9629\") " pod="openshift-multus/network-metrics-daemon-mpnqv" Jul 23 04:39:27.181138 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.179206 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-var-lib-kubelet\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.181138 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.179260 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-host-slash\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.181138 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.179344 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-hostroot\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.181138 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.179402 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-system-cni-dir\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.181138 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.179414 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-run-openvswitch\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.181864 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.179436 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/43514e24-f4a7-4dfc-b45d-12d28e0a4bfc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9nzz9\" (UID: \"43514e24-f4a7-4dfc-b45d-12d28e0a4bfc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9nzz9" Jul 23 04:39:27.181864 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.179440 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0bf5e605-388c-4f50-8716-c350190d11d9-host-run-netns\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.181864 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.179462 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/43514e24-f4a7-4dfc-b45d-12d28e0a4bfc-device-dir\") pod \"aws-ebs-csi-driver-node-9nzz9\" (UID: \"43514e24-f4a7-4dfc-b45d-12d28e0a4bfc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9nzz9" Jul 23 04:39:27.181864 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.179475 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/67a7fb5e-8e74-4275-8abd-56f340394bd0-iptables-alerter-script\") pod \"iptables-alerter-htlgx\" (UID: \"67a7fb5e-8e74-4275-8abd-56f340394bd0\") " pod="openshift-network-operator/iptables-alerter-htlgx" Jul 23 04:39:27.181864 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.179482 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-var-lib-kubelet\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.181864 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.179561 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/43514e24-f4a7-4dfc-b45d-12d28e0a4bfc-device-dir\") pod \"aws-ebs-csi-driver-node-9nzz9\" (UID: \"43514e24-f4a7-4dfc-b45d-12d28e0a4bfc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9nzz9" Jul 23 04:39:27.181864 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.179533 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-host-slash\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.181864 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.179599 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b3007292-2534-4dc0-82a3-41d6a7d34c0c-run-openvswitch\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.181864 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.179638 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/43514e24-f4a7-4dfc-b45d-12d28e0a4bfc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9nzz9\" (UID: \"43514e24-f4a7-4dfc-b45d-12d28e0a4bfc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9nzz9" Jul 23 04:39:27.181864 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.180184 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-tmp\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.181864 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.180436 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b3007292-2534-4dc0-82a3-41d6a7d34c0c-ovn-node-metrics-cert\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.181864 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.180672 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ba17063f-a4cc-4378-afe1-6b4aad36adef-agent-certs\") pod \"konnectivity-agent-zhq6d\" (UID: \"ba17063f-a4cc-4378-afe1-6b4aad36adef\") " pod="kube-system/konnectivity-agent-zhq6d" Jul 23 04:39:27.181864 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.181418 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-etc-tuned\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.182995 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.182769 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 04:39:27.186979 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.186944 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mbkq\" (UniqueName: \"kubernetes.io/projected/0bf5e605-388c-4f50-8716-c350190d11d9-kube-api-access-8mbkq\") pod \"multus-w7d9f\" (UID: \"0bf5e605-388c-4f50-8716-c350190d11d9\") " pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.187100 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.186995 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x5h7\" (UniqueName: \"kubernetes.io/projected/32fdf9c3-bed9-4d00-82f9-18e6f93dbf88-kube-api-access-4x5h7\") pod \"tuned-q2kph\" (UID: \"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88\") " pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.187100 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.187077 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmx8b\" (UniqueName: \"kubernetes.io/projected/c49b7ef0-9710-41cd-9d89-145654e92fc6-kube-api-access-wmx8b\") pod \"multus-additional-cni-plugins-zzgwf\" (UID: \"c49b7ef0-9710-41cd-9d89-145654e92fc6\") " pod="openshift-multus/multus-additional-cni-plugins-zzgwf" Jul 23 04:39:27.187227 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.187111 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75z5n\" (UniqueName: \"kubernetes.io/projected/d69de93b-d93f-4f10-b618-0cdcb545197a-kube-api-access-75z5n\") pod \"node-ca-grlbc\" (UID: \"d69de93b-d93f-4f10-b618-0cdcb545197a\") " pod="openshift-image-registry/node-ca-grlbc" Jul 23 04:39:27.187227 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.187154 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svpl8\" (UniqueName: \"kubernetes.io/projected/67a7fb5e-8e74-4275-8abd-56f340394bd0-kube-api-access-svpl8\") pod \"iptables-alerter-htlgx\" (UID: \"67a7fb5e-8e74-4275-8abd-56f340394bd0\") " pod="openshift-network-operator/iptables-alerter-htlgx" Jul 23 04:39:27.187875 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.187824 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tk595\" (UniqueName: \"kubernetes.io/projected/b3007292-2534-4dc0-82a3-41d6a7d34c0c-kube-api-access-tk595\") pod \"ovnkube-node-vphbq\" (UID: \"b3007292-2534-4dc0-82a3-41d6a7d34c0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.189216 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.189193 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7szxj\" (UniqueName: \"kubernetes.io/projected/43514e24-f4a7-4dfc-b45d-12d28e0a4bfc-kube-api-access-7szxj\") pod \"aws-ebs-csi-driver-node-9nzz9\" (UID: \"43514e24-f4a7-4dfc-b45d-12d28e0a4bfc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9nzz9" Jul 23 04:39:27.206441 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.206389 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-96.ec2.internal" event={"ID":"62fa0ee5fa616af59e13c3ed7949ae7f","Type":"ContainerStarted","Data":"87e5c701ab7c63c489ee77688e4a4034246b728455adfda8238a1c75186d55f8"} Jul 23 04:39:27.207461 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.207437 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-96.ec2.internal" event={"ID":"5666b1587a613009c403213e61aac89c","Type":"ContainerStarted","Data":"1ade9c247fff30945033a9ebe362e8daf890ef89d3737fa9cd44e506e4fda96b"} Jul 23 04:39:27.280052 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.280020 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/555665cd-6364-4d9f-8326-4973b140cc64-dbus\") pod \"global-pull-secret-syncer-cbzpv\" (UID: \"555665cd-6364-4d9f-8326-4973b140cc64\") " pod="kube-system/global-pull-secret-syncer-cbzpv" Jul 23 04:39:27.280052 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.280058 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/555665cd-6364-4d9f-8326-4973b140cc64-kubelet-config\") pod \"global-pull-secret-syncer-cbzpv\" (UID: \"555665cd-6364-4d9f-8326-4973b140cc64\") " pod="kube-system/global-pull-secret-syncer-cbzpv" Jul 23 04:39:27.280286 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.280079 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xncd8\" (UniqueName: \"kubernetes.io/projected/dab60874-8b3b-411d-b58e-4ec857bc9629-kube-api-access-xncd8\") pod \"network-metrics-daemon-mpnqv\" (UID: \"dab60874-8b3b-411d-b58e-4ec857bc9629\") " pod="openshift-multus/network-metrics-daemon-mpnqv" Jul 23 04:39:27.280286 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.280107 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czjdx\" (UniqueName: \"kubernetes.io/projected/6346ee84-530c-41f1-9f8a-6d83f79daa5e-kube-api-access-czjdx\") pod \"network-check-target-7dsv7\" (UID: \"6346ee84-530c-41f1-9f8a-6d83f79daa5e\") " pod="openshift-network-diagnostics/network-check-target-7dsv7" Jul 23 04:39:27.280286 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.280138 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9a49f0e-338a-47ac-90f7-281dae5538e4-tmp-dir\") pod \"node-resolver-57bmk\" (UID: \"b9a49f0e-338a-47ac-90f7-281dae5538e4\") " pod="openshift-dns/node-resolver-57bmk" Jul 23 04:39:27.280286 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.280155 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/555665cd-6364-4d9f-8326-4973b140cc64-dbus\") pod \"global-pull-secret-syncer-cbzpv\" (UID: \"555665cd-6364-4d9f-8326-4973b140cc64\") " pod="kube-system/global-pull-secret-syncer-cbzpv" Jul 23 04:39:27.280286 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.280180 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dab60874-8b3b-411d-b58e-4ec857bc9629-metrics-certs\") pod \"network-metrics-daemon-mpnqv\" (UID: \"dab60874-8b3b-411d-b58e-4ec857bc9629\") " pod="openshift-multus/network-metrics-daemon-mpnqv" Jul 23 04:39:27.280286 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.280205 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vchx\" (UniqueName: \"kubernetes.io/projected/b9a49f0e-338a-47ac-90f7-281dae5538e4-kube-api-access-2vchx\") pod \"node-resolver-57bmk\" (UID: \"b9a49f0e-338a-47ac-90f7-281dae5538e4\") " pod="openshift-dns/node-resolver-57bmk" Jul 23 04:39:27.280286 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.280234 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/555665cd-6364-4d9f-8326-4973b140cc64-original-pull-secret\") pod \"global-pull-secret-syncer-cbzpv\" (UID: \"555665cd-6364-4d9f-8326-4973b140cc64\") " pod="kube-system/global-pull-secret-syncer-cbzpv" Jul 23 04:39:27.280286 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.280259 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b9a49f0e-338a-47ac-90f7-281dae5538e4-hosts-file\") pod \"node-resolver-57bmk\" (UID: \"b9a49f0e-338a-47ac-90f7-281dae5538e4\") " pod="openshift-dns/node-resolver-57bmk" Jul 23 04:39:27.280710 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.280329 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b9a49f0e-338a-47ac-90f7-281dae5538e4-hosts-file\") pod \"node-resolver-57bmk\" (UID: \"b9a49f0e-338a-47ac-90f7-281dae5538e4\") " pod="openshift-dns/node-resolver-57bmk" Jul 23 04:39:27.280710 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:27.280409 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 04:39:27.280710 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:27.280438 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 04:39:27.280710 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:27.280504 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/555665cd-6364-4d9f-8326-4973b140cc64-original-pull-secret podName:555665cd-6364-4d9f-8326-4973b140cc64 nodeName:}" failed. No retries permitted until 2026-07-23 04:39:27.780465922 +0000 UTC m=+3.122978809 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/555665cd-6364-4d9f-8326-4973b140cc64-original-pull-secret") pod "global-pull-secret-syncer-cbzpv" (UID: "555665cd-6364-4d9f-8326-4973b140cc64") : object "kube-system"/"original-pull-secret" not registered Jul 23 04:39:27.280710 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:27.280519 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dab60874-8b3b-411d-b58e-4ec857bc9629-metrics-certs podName:dab60874-8b3b-411d-b58e-4ec857bc9629 nodeName:}" failed. No retries permitted until 2026-07-23 04:39:27.78051277 +0000 UTC m=+3.123025637 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dab60874-8b3b-411d-b58e-4ec857bc9629-metrics-certs") pod "network-metrics-daemon-mpnqv" (UID: "dab60874-8b3b-411d-b58e-4ec857bc9629") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 04:39:27.280710 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.280549 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9a49f0e-338a-47ac-90f7-281dae5538e4-tmp-dir\") pod \"node-resolver-57bmk\" (UID: \"b9a49f0e-338a-47ac-90f7-281dae5538e4\") " pod="openshift-dns/node-resolver-57bmk" Jul 23 04:39:27.280710 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.280588 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/555665cd-6364-4d9f-8326-4973b140cc64-kubelet-config\") pod \"global-pull-secret-syncer-cbzpv\" (UID: \"555665cd-6364-4d9f-8326-4973b140cc64\") " pod="kube-system/global-pull-secret-syncer-cbzpv" Jul 23 04:39:27.291293 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:27.291225 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 04:39:27.291293 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:27.291253 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 04:39:27.291293 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:27.291267 2562 projected.go:196] Error preparing data for projected volume kube-api-access-czjdx for pod openshift-network-diagnostics/network-check-target-7dsv7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 04:39:27.291512 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:27.291334 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6346ee84-530c-41f1-9f8a-6d83f79daa5e-kube-api-access-czjdx podName:6346ee84-530c-41f1-9f8a-6d83f79daa5e nodeName:}" failed. No retries permitted until 2026-07-23 04:39:27.791315679 +0000 UTC m=+3.133828561 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-czjdx" (UniqueName: "kubernetes.io/projected/6346ee84-530c-41f1-9f8a-6d83f79daa5e-kube-api-access-czjdx") pod "network-check-target-7dsv7" (UID: "6346ee84-530c-41f1-9f8a-6d83f79daa5e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 04:39:27.295072 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.295046 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vchx\" (UniqueName: \"kubernetes.io/projected/b9a49f0e-338a-47ac-90f7-281dae5538e4-kube-api-access-2vchx\") pod \"node-resolver-57bmk\" (UID: \"b9a49f0e-338a-47ac-90f7-281dae5538e4\") " pod="openshift-dns/node-resolver-57bmk" Jul 23 04:39:27.295072 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.295067 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xncd8\" (UniqueName: \"kubernetes.io/projected/dab60874-8b3b-411d-b58e-4ec857bc9629-kube-api-access-xncd8\") pod \"network-metrics-daemon-mpnqv\" (UID: \"dab60874-8b3b-411d-b58e-4ec857bc9629\") " pod="openshift-multus/network-metrics-daemon-mpnqv" Jul 23 04:39:27.362024 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.361981 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-grlbc" Jul 23 04:39:27.372588 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.372553 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9nzz9" Jul 23 04:39:27.383817 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.383780 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-htlgx" Jul 23 04:39:27.390736 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.390711 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zhq6d" Jul 23 04:39:27.398583 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.398559 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-w7d9f" Jul 23 04:39:27.406187 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.406163 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-q2kph" Jul 23 04:39:27.418194 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.418159 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:27.424442 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.424417 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zzgwf" Jul 23 04:39:27.429808 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.429788 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-57bmk" Jul 23 04:39:27.783158 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.783121 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dab60874-8b3b-411d-b58e-4ec857bc9629-metrics-certs\") pod \"network-metrics-daemon-mpnqv\" (UID: \"dab60874-8b3b-411d-b58e-4ec857bc9629\") " pod="openshift-multus/network-metrics-daemon-mpnqv" Jul 23 04:39:27.783351 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.783179 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/555665cd-6364-4d9f-8326-4973b140cc64-original-pull-secret\") pod \"global-pull-secret-syncer-cbzpv\" (UID: \"555665cd-6364-4d9f-8326-4973b140cc64\") " pod="kube-system/global-pull-secret-syncer-cbzpv" Jul 23 04:39:27.783351 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:27.783289 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 04:39:27.783351 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:27.783307 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 04:39:27.783351 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:27.783352 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dab60874-8b3b-411d-b58e-4ec857bc9629-metrics-certs podName:dab60874-8b3b-411d-b58e-4ec857bc9629 nodeName:}" failed. No retries permitted until 2026-07-23 04:39:28.783333501 +0000 UTC m=+4.125846390 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dab60874-8b3b-411d-b58e-4ec857bc9629-metrics-certs") pod "network-metrics-daemon-mpnqv" (UID: "dab60874-8b3b-411d-b58e-4ec857bc9629") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 04:39:27.783534 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:27.783371 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/555665cd-6364-4d9f-8326-4973b140cc64-original-pull-secret podName:555665cd-6364-4d9f-8326-4973b140cc64 nodeName:}" failed. No retries permitted until 2026-07-23 04:39:28.78336224 +0000 UTC m=+4.125875111 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/555665cd-6364-4d9f-8326-4973b140cc64-original-pull-secret") pod "global-pull-secret-syncer-cbzpv" (UID: "555665cd-6364-4d9f-8326-4973b140cc64") : object "kube-system"/"original-pull-secret" not registered Jul 23 04:39:27.884512 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:27.884475 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czjdx\" (UniqueName: \"kubernetes.io/projected/6346ee84-530c-41f1-9f8a-6d83f79daa5e-kube-api-access-czjdx\") pod \"network-check-target-7dsv7\" (UID: \"6346ee84-530c-41f1-9f8a-6d83f79daa5e\") " pod="openshift-network-diagnostics/network-check-target-7dsv7" Jul 23 04:39:27.884701 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:27.884671 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 04:39:27.884701 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:27.884692 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 04:39:27.884794 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:27.884705 2562 projected.go:196] Error preparing data for projected volume kube-api-access-czjdx for pod openshift-network-diagnostics/network-check-target-7dsv7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 04:39:27.884794 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:27.884767 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6346ee84-530c-41f1-9f8a-6d83f79daa5e-kube-api-access-czjdx podName:6346ee84-530c-41f1-9f8a-6d83f79daa5e nodeName:}" failed. No retries permitted until 2026-07-23 04:39:28.884751554 +0000 UTC m=+4.227264636 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-czjdx" (UniqueName: "kubernetes.io/projected/6346ee84-530c-41f1-9f8a-6d83f79daa5e-kube-api-access-czjdx") pod "network-check-target-7dsv7" (UID: "6346ee84-530c-41f1-9f8a-6d83f79daa5e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 04:39:28.063758 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:28.063727 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bf5e605_388c_4f50_8716_c350190d11d9.slice/crio-16dc15781acc5190d8da1ac40ce2d9df986f7aeae1bfd9868c7f4da629706513 WatchSource:0}: Error finding container 16dc15781acc5190d8da1ac40ce2d9df986f7aeae1bfd9868c7f4da629706513: Status 404 returned error can't find the container with id 16dc15781acc5190d8da1ac40ce2d9df986f7aeae1bfd9868c7f4da629706513 Jul 23 04:39:28.064761 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:28.064712 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43514e24_f4a7_4dfc_b45d_12d28e0a4bfc.slice/crio-e9abd72c0303c9b636d855c80a6dd57c71c0fc96133e1dcf130b7202044613bf WatchSource:0}: Error finding container e9abd72c0303c9b636d855c80a6dd57c71c0fc96133e1dcf130b7202044613bf: Status 404 returned error can't find the container with id e9abd72c0303c9b636d855c80a6dd57c71c0fc96133e1dcf130b7202044613bf Jul 23 04:39:28.065964 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:28.065928 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba17063f_a4cc_4378_afe1_6b4aad36adef.slice/crio-d7f6ad7ac201c8751f0b65c9a98f4b2a3597772943894079f3249a3cbb508f6e WatchSource:0}: Error finding container d7f6ad7ac201c8751f0b65c9a98f4b2a3597772943894079f3249a3cbb508f6e: Status 404 returned error can't find the container with id d7f6ad7ac201c8751f0b65c9a98f4b2a3597772943894079f3249a3cbb508f6e Jul 23 04:39:28.066283 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:28.066123 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc49b7ef0_9710_41cd_9d89_145654e92fc6.slice/crio-66621d8feb32c5637dc60c70506d9097fbd208d540f428653a10b52b730e7da5 WatchSource:0}: Error finding container 66621d8feb32c5637dc60c70506d9097fbd208d540f428653a10b52b730e7da5: Status 404 returned error can't find the container with id 66621d8feb32c5637dc60c70506d9097fbd208d540f428653a10b52b730e7da5 Jul 23 04:39:28.069534 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:28.069512 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32fdf9c3_bed9_4d00_82f9_18e6f93dbf88.slice/crio-39deeb3fba1122e3098bbac153257f64d60bf526bb583c6d2dcfbe1be6671bfe WatchSource:0}: Error finding container 39deeb3fba1122e3098bbac153257f64d60bf526bb583c6d2dcfbe1be6671bfe: Status 404 returned error can't find the container with id 39deeb3fba1122e3098bbac153257f64d60bf526bb583c6d2dcfbe1be6671bfe Jul 23 04:39:28.070524 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:28.070504 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67a7fb5e_8e74_4275_8abd_56f340394bd0.slice/crio-d480f6a025e60dc6f0652ffb88820787f441c396dc661bbeb3d4732d936201d2 WatchSource:0}: Error finding container d480f6a025e60dc6f0652ffb88820787f441c396dc661bbeb3d4732d936201d2: Status 404 returned error can't find the container with id d480f6a025e60dc6f0652ffb88820787f441c396dc661bbeb3d4732d936201d2 Jul 23 04:39:28.072697 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:28.072676 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd69de93b_d93f_4f10_b618_0cdcb545197a.slice/crio-1a96ac203f0687aa4748dda0b30838381a44612f1cff4e18229b5a1f078c9909 WatchSource:0}: Error finding container 1a96ac203f0687aa4748dda0b30838381a44612f1cff4e18229b5a1f078c9909: Status 404 returned error can't find the container with id 1a96ac203f0687aa4748dda0b30838381a44612f1cff4e18229b5a1f078c9909 Jul 23 04:39:28.073625 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:28.073606 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9a49f0e_338a_47ac_90f7_281dae5538e4.slice/crio-39d36ab19f7a23d08cfdc7868c6a6587a52bd91dddbc1a8a3155a98b7822a007 WatchSource:0}: Error finding container 39d36ab19f7a23d08cfdc7868c6a6587a52bd91dddbc1a8a3155a98b7822a007: Status 404 returned error can't find the container with id 39d36ab19f7a23d08cfdc7868c6a6587a52bd91dddbc1a8a3155a98b7822a007 Jul 23 04:39:28.075225 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:39:28.075157 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3007292_2534_4dc0_82a3_41d6a7d34c0c.slice/crio-0c02a4cc88853f4f096045c03412ed77357fb8261f4bf737a54ecf20f1898cd8 WatchSource:0}: Error finding container 0c02a4cc88853f4f096045c03412ed77357fb8261f4bf737a54ecf20f1898cd8: Status 404 returned error can't find the container with id 0c02a4cc88853f4f096045c03412ed77357fb8261f4bf737a54ecf20f1898cd8 Jul 23 04:39:28.107137 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:28.107104 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 04:34:27 +0000 UTC" deadline="2028-01-18 09:01:18.426467182 +0000 UTC" Jul 23 04:39:28.107249 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:28.107139 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13060h21m50.319333325s" Jul 23 04:39:28.201886 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:28.201621 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbzpv" Jul 23 04:39:28.202005 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:28.201668 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mpnqv" Jul 23 04:39:28.202005 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:28.201992 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbzpv" podUID="555665cd-6364-4d9f-8326-4973b140cc64" Jul 23 04:39:28.202167 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:28.202145 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mpnqv" podUID="dab60874-8b3b-411d-b58e-4ec857bc9629" Jul 23 04:39:28.210711 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:28.210685 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-96.ec2.internal" event={"ID":"5666b1587a613009c403213e61aac89c","Type":"ContainerStarted","Data":"6e28c70c8d74cb879f73326f3c4a89a98fbb85bc1aa69c28dce9ba445d61ccb2"} Jul 23 04:39:28.212082 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:28.212060 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" event={"ID":"b3007292-2534-4dc0-82a3-41d6a7d34c0c","Type":"ContainerStarted","Data":"0c02a4cc88853f4f096045c03412ed77357fb8261f4bf737a54ecf20f1898cd8"} Jul 23 04:39:28.213255 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:28.213235 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-grlbc" event={"ID":"d69de93b-d93f-4f10-b618-0cdcb545197a","Type":"ContainerStarted","Data":"1a96ac203f0687aa4748dda0b30838381a44612f1cff4e18229b5a1f078c9909"} Jul 23 04:39:28.214382 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:28.214362 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-57bmk" event={"ID":"b9a49f0e-338a-47ac-90f7-281dae5538e4","Type":"ContainerStarted","Data":"39d36ab19f7a23d08cfdc7868c6a6587a52bd91dddbc1a8a3155a98b7822a007"} Jul 23 04:39:28.215593 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:28.215576 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-q2kph" event={"ID":"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88","Type":"ContainerStarted","Data":"39deeb3fba1122e3098bbac153257f64d60bf526bb583c6d2dcfbe1be6671bfe"} Jul 23 04:39:28.216771 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:28.216745 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-w7d9f" event={"ID":"0bf5e605-388c-4f50-8716-c350190d11d9","Type":"ContainerStarted","Data":"16dc15781acc5190d8da1ac40ce2d9df986f7aeae1bfd9868c7f4da629706513"} Jul 23 04:39:28.217950 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:28.217919 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-htlgx" event={"ID":"67a7fb5e-8e74-4275-8abd-56f340394bd0","Type":"ContainerStarted","Data":"d480f6a025e60dc6f0652ffb88820787f441c396dc661bbeb3d4732d936201d2"} Jul 23 04:39:28.222324 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:28.222289 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zzgwf" event={"ID":"c49b7ef0-9710-41cd-9d89-145654e92fc6","Type":"ContainerStarted","Data":"66621d8feb32c5637dc60c70506d9097fbd208d540f428653a10b52b730e7da5"} Jul 23 04:39:28.223323 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:28.223301 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zhq6d" event={"ID":"ba17063f-a4cc-4378-afe1-6b4aad36adef","Type":"ContainerStarted","Data":"d7f6ad7ac201c8751f0b65c9a98f4b2a3597772943894079f3249a3cbb508f6e"} Jul 23 04:39:28.224467 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:28.224448 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9nzz9" event={"ID":"43514e24-f4a7-4dfc-b45d-12d28e0a4bfc","Type":"ContainerStarted","Data":"e9abd72c0303c9b636d855c80a6dd57c71c0fc96133e1dcf130b7202044613bf"} Jul 23 04:39:28.226003 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:28.225964 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-96.ec2.internal" podStartSLOduration=2.225948774 podStartE2EDuration="2.225948774s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 04:39:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:39:28.225454934 +0000 UTC m=+3.567967822" watchObservedRunningTime="2026-07-23 04:39:28.225948774 +0000 UTC m=+3.568461664" Jul 23 04:39:28.790629 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:28.789868 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dab60874-8b3b-411d-b58e-4ec857bc9629-metrics-certs\") pod \"network-metrics-daemon-mpnqv\" (UID: \"dab60874-8b3b-411d-b58e-4ec857bc9629\") " pod="openshift-multus/network-metrics-daemon-mpnqv" Jul 23 04:39:28.790629 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:28.789937 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/555665cd-6364-4d9f-8326-4973b140cc64-original-pull-secret\") pod \"global-pull-secret-syncer-cbzpv\" (UID: \"555665cd-6364-4d9f-8326-4973b140cc64\") " pod="kube-system/global-pull-secret-syncer-cbzpv" Jul 23 04:39:28.790629 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:28.790098 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 04:39:28.790629 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:28.790161 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/555665cd-6364-4d9f-8326-4973b140cc64-original-pull-secret podName:555665cd-6364-4d9f-8326-4973b140cc64 nodeName:}" failed. No retries permitted until 2026-07-23 04:39:30.790141304 +0000 UTC m=+6.132654175 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/555665cd-6364-4d9f-8326-4973b140cc64-original-pull-secret") pod "global-pull-secret-syncer-cbzpv" (UID: "555665cd-6364-4d9f-8326-4973b140cc64") : object "kube-system"/"original-pull-secret" not registered Jul 23 04:39:28.790629 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:28.790538 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 04:39:28.790629 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:28.790588 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dab60874-8b3b-411d-b58e-4ec857bc9629-metrics-certs podName:dab60874-8b3b-411d-b58e-4ec857bc9629 nodeName:}" failed. No retries permitted until 2026-07-23 04:39:30.790572097 +0000 UTC m=+6.133084968 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dab60874-8b3b-411d-b58e-4ec857bc9629-metrics-certs") pod "network-metrics-daemon-mpnqv" (UID: "dab60874-8b3b-411d-b58e-4ec857bc9629") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 04:39:28.891184 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:28.891143 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czjdx\" (UniqueName: \"kubernetes.io/projected/6346ee84-530c-41f1-9f8a-6d83f79daa5e-kube-api-access-czjdx\") pod \"network-check-target-7dsv7\" (UID: \"6346ee84-530c-41f1-9f8a-6d83f79daa5e\") " pod="openshift-network-diagnostics/network-check-target-7dsv7" Jul 23 04:39:28.891396 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:28.891378 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 04:39:28.891477 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:28.891403 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 04:39:28.891477 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:28.891418 2562 projected.go:196] Error preparing data for projected volume kube-api-access-czjdx for pod openshift-network-diagnostics/network-check-target-7dsv7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 04:39:28.891580 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:28.891478 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6346ee84-530c-41f1-9f8a-6d83f79daa5e-kube-api-access-czjdx podName:6346ee84-530c-41f1-9f8a-6d83f79daa5e nodeName:}" failed. No retries permitted until 2026-07-23 04:39:30.891460936 +0000 UTC m=+6.233973806 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-czjdx" (UniqueName: "kubernetes.io/projected/6346ee84-530c-41f1-9f8a-6d83f79daa5e-kube-api-access-czjdx") pod "network-check-target-7dsv7" (UID: "6346ee84-530c-41f1-9f8a-6d83f79daa5e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 04:39:29.107711 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:29.107646 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 04:34:27 +0000 UTC" deadline="2028-03-11 02:41:55.452375403 +0000 UTC" Jul 23 04:39:29.107711 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:29.107708 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14326h2m26.34467112s" Jul 23 04:39:29.201716 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:29.201615 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dsv7" Jul 23 04:39:29.201916 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:29.201854 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dsv7" podUID="6346ee84-530c-41f1-9f8a-6d83f79daa5e" Jul 23 04:39:29.256158 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:29.254989 2562 generic.go:356] "Generic (PLEG): container finished" podID="62fa0ee5fa616af59e13c3ed7949ae7f" containerID="ce6ed87cc59bf2e2875f1e98cee77f7702a0b8280a90ddd082993f28bcfbc7b9" exitCode=0 Jul 23 04:39:29.256158 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:29.255944 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-96.ec2.internal" event={"ID":"62fa0ee5fa616af59e13c3ed7949ae7f","Type":"ContainerDied","Data":"ce6ed87cc59bf2e2875f1e98cee77f7702a0b8280a90ddd082993f28bcfbc7b9"} Jul 23 04:39:30.202310 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:30.202276 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbzpv" Jul 23 04:39:30.202787 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:30.202289 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mpnqv" Jul 23 04:39:30.202787 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:30.202416 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbzpv" podUID="555665cd-6364-4d9f-8326-4973b140cc64" Jul 23 04:39:30.202787 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:30.202452 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mpnqv" podUID="dab60874-8b3b-411d-b58e-4ec857bc9629" Jul 23 04:39:30.277489 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:30.277428 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-96.ec2.internal" event={"ID":"62fa0ee5fa616af59e13c3ed7949ae7f","Type":"ContainerStarted","Data":"5a3bedb905ca2649a8251e5b13447734aa7c0ee4b5b8dacdd5dd80ff36922c53"} Jul 23 04:39:30.296543 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:30.296484 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-96.ec2.internal" podStartSLOduration=4.296463155 podStartE2EDuration="4.296463155s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 04:39:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:39:30.295800349 +0000 UTC m=+5.638313244" watchObservedRunningTime="2026-07-23 04:39:30.296463155 +0000 UTC m=+5.638976044" Jul 23 04:39:30.807654 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:30.807613 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dab60874-8b3b-411d-b58e-4ec857bc9629-metrics-certs\") pod \"network-metrics-daemon-mpnqv\" (UID: \"dab60874-8b3b-411d-b58e-4ec857bc9629\") " pod="openshift-multus/network-metrics-daemon-mpnqv" Jul 23 04:39:30.807858 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:30.807674 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/555665cd-6364-4d9f-8326-4973b140cc64-original-pull-secret\") pod \"global-pull-secret-syncer-cbzpv\" (UID: \"555665cd-6364-4d9f-8326-4973b140cc64\") " pod="kube-system/global-pull-secret-syncer-cbzpv" Jul 23 04:39:30.807858 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:30.807799 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 04:39:30.807984 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:30.807878 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/555665cd-6364-4d9f-8326-4973b140cc64-original-pull-secret podName:555665cd-6364-4d9f-8326-4973b140cc64 nodeName:}" failed. No retries permitted until 2026-07-23 04:39:34.807859433 +0000 UTC m=+10.150372302 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/555665cd-6364-4d9f-8326-4973b140cc64-original-pull-secret") pod "global-pull-secret-syncer-cbzpv" (UID: "555665cd-6364-4d9f-8326-4973b140cc64") : object "kube-system"/"original-pull-secret" not registered Jul 23 04:39:30.808292 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:30.808271 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 04:39:30.808362 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:30.808325 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dab60874-8b3b-411d-b58e-4ec857bc9629-metrics-certs podName:dab60874-8b3b-411d-b58e-4ec857bc9629 nodeName:}" failed. No retries permitted until 2026-07-23 04:39:34.80831035 +0000 UTC m=+10.150823219 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dab60874-8b3b-411d-b58e-4ec857bc9629-metrics-certs") pod "network-metrics-daemon-mpnqv" (UID: "dab60874-8b3b-411d-b58e-4ec857bc9629") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 04:39:30.908673 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:30.908637 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czjdx\" (UniqueName: \"kubernetes.io/projected/6346ee84-530c-41f1-9f8a-6d83f79daa5e-kube-api-access-czjdx\") pod \"network-check-target-7dsv7\" (UID: \"6346ee84-530c-41f1-9f8a-6d83f79daa5e\") " pod="openshift-network-diagnostics/network-check-target-7dsv7" Jul 23 04:39:30.908965 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:30.908814 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 04:39:30.908965 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:30.908852 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 04:39:30.908965 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:30.908867 2562 projected.go:196] Error preparing data for projected volume kube-api-access-czjdx for pod openshift-network-diagnostics/network-check-target-7dsv7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 04:39:30.908965 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:30.908923 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6346ee84-530c-41f1-9f8a-6d83f79daa5e-kube-api-access-czjdx podName:6346ee84-530c-41f1-9f8a-6d83f79daa5e nodeName:}" failed. No retries permitted until 2026-07-23 04:39:34.908903938 +0000 UTC m=+10.251416807 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-czjdx" (UniqueName: "kubernetes.io/projected/6346ee84-530c-41f1-9f8a-6d83f79daa5e-kube-api-access-czjdx") pod "network-check-target-7dsv7" (UID: "6346ee84-530c-41f1-9f8a-6d83f79daa5e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 04:39:31.204047 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:31.203713 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dsv7" Jul 23 04:39:31.204047 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:31.203818 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dsv7" podUID="6346ee84-530c-41f1-9f8a-6d83f79daa5e" Jul 23 04:39:32.202794 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:32.202759 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbzpv" Jul 23 04:39:32.202993 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:32.202907 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbzpv" podUID="555665cd-6364-4d9f-8326-4973b140cc64" Jul 23 04:39:32.204051 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:32.203312 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mpnqv" Jul 23 04:39:32.204051 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:32.203406 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mpnqv" podUID="dab60874-8b3b-411d-b58e-4ec857bc9629" Jul 23 04:39:33.201576 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:33.201541 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dsv7" Jul 23 04:39:33.201736 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:33.201676 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dsv7" podUID="6346ee84-530c-41f1-9f8a-6d83f79daa5e" Jul 23 04:39:34.201868 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:34.201611 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mpnqv" Jul 23 04:39:34.201868 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:34.201760 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mpnqv" podUID="dab60874-8b3b-411d-b58e-4ec857bc9629" Jul 23 04:39:34.201868 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:34.201804 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbzpv" Jul 23 04:39:34.202380 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:34.201890 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbzpv" podUID="555665cd-6364-4d9f-8326-4973b140cc64" Jul 23 04:39:34.842847 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:34.842795 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dab60874-8b3b-411d-b58e-4ec857bc9629-metrics-certs\") pod \"network-metrics-daemon-mpnqv\" (UID: \"dab60874-8b3b-411d-b58e-4ec857bc9629\") " pod="openshift-multus/network-metrics-daemon-mpnqv" Jul 23 04:39:34.843026 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:34.842866 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/555665cd-6364-4d9f-8326-4973b140cc64-original-pull-secret\") pod \"global-pull-secret-syncer-cbzpv\" (UID: \"555665cd-6364-4d9f-8326-4973b140cc64\") " pod="kube-system/global-pull-secret-syncer-cbzpv" Jul 23 04:39:34.843026 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:34.842957 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 04:39:34.843026 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:34.842997 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 04:39:34.843026 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:34.843024 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dab60874-8b3b-411d-b58e-4ec857bc9629-metrics-certs podName:dab60874-8b3b-411d-b58e-4ec857bc9629 nodeName:}" failed. No retries permitted until 2026-07-23 04:39:42.843005199 +0000 UTC m=+18.185518083 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dab60874-8b3b-411d-b58e-4ec857bc9629-metrics-certs") pod "network-metrics-daemon-mpnqv" (UID: "dab60874-8b3b-411d-b58e-4ec857bc9629") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 04:39:34.843217 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:34.843050 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/555665cd-6364-4d9f-8326-4973b140cc64-original-pull-secret podName:555665cd-6364-4d9f-8326-4973b140cc64 nodeName:}" failed. No retries permitted until 2026-07-23 04:39:42.84303386 +0000 UTC m=+18.185546746 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/555665cd-6364-4d9f-8326-4973b140cc64-original-pull-secret") pod "global-pull-secret-syncer-cbzpv" (UID: "555665cd-6364-4d9f-8326-4973b140cc64") : object "kube-system"/"original-pull-secret" not registered Jul 23 04:39:34.944244 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:34.944175 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czjdx\" (UniqueName: \"kubernetes.io/projected/6346ee84-530c-41f1-9f8a-6d83f79daa5e-kube-api-access-czjdx\") pod \"network-check-target-7dsv7\" (UID: \"6346ee84-530c-41f1-9f8a-6d83f79daa5e\") " pod="openshift-network-diagnostics/network-check-target-7dsv7" Jul 23 04:39:34.944423 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:34.944363 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 04:39:34.944423 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:34.944394 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 04:39:34.944423 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:34.944409 2562 projected.go:196] Error preparing data for projected volume kube-api-access-czjdx for pod openshift-network-diagnostics/network-check-target-7dsv7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 04:39:34.944521 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:34.944485 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6346ee84-530c-41f1-9f8a-6d83f79daa5e-kube-api-access-czjdx podName:6346ee84-530c-41f1-9f8a-6d83f79daa5e nodeName:}" failed. No retries permitted until 2026-07-23 04:39:42.944464155 +0000 UTC m=+18.286977028 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-czjdx" (UniqueName: "kubernetes.io/projected/6346ee84-530c-41f1-9f8a-6d83f79daa5e-kube-api-access-czjdx") pod "network-check-target-7dsv7" (UID: "6346ee84-530c-41f1-9f8a-6d83f79daa5e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 04:39:35.203814 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:35.203739 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dsv7" Jul 23 04:39:35.204402 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:35.203887 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dsv7" podUID="6346ee84-530c-41f1-9f8a-6d83f79daa5e" Jul 23 04:39:36.202372 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:36.202334 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbzpv" Jul 23 04:39:36.202553 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:36.202355 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mpnqv" Jul 23 04:39:36.202553 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:36.202485 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbzpv" podUID="555665cd-6364-4d9f-8326-4973b140cc64" Jul 23 04:39:36.202553 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:36.202542 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mpnqv" podUID="dab60874-8b3b-411d-b58e-4ec857bc9629" Jul 23 04:39:37.203987 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:37.202096 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dsv7" Jul 23 04:39:37.203987 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:37.203806 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dsv7" podUID="6346ee84-530c-41f1-9f8a-6d83f79daa5e" Jul 23 04:39:38.201996 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:38.201962 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbzpv" Jul 23 04:39:38.202167 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:38.201969 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mpnqv" Jul 23 04:39:38.202167 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:38.202113 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbzpv" podUID="555665cd-6364-4d9f-8326-4973b140cc64" Jul 23 04:39:38.202264 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:38.202207 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mpnqv" podUID="dab60874-8b3b-411d-b58e-4ec857bc9629" Jul 23 04:39:39.201992 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:39.201953 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dsv7" Jul 23 04:39:39.202456 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:39.202094 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dsv7" podUID="6346ee84-530c-41f1-9f8a-6d83f79daa5e" Jul 23 04:39:40.202261 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:40.202220 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mpnqv" Jul 23 04:39:40.202705 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:40.202220 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbzpv" Jul 23 04:39:40.202705 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:40.202389 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mpnqv" podUID="dab60874-8b3b-411d-b58e-4ec857bc9629" Jul 23 04:39:40.202705 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:40.202460 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbzpv" podUID="555665cd-6364-4d9f-8326-4973b140cc64" Jul 23 04:39:41.202103 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:41.202070 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dsv7" Jul 23 04:39:41.202285 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:41.202188 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dsv7" podUID="6346ee84-530c-41f1-9f8a-6d83f79daa5e" Jul 23 04:39:42.202152 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:42.202108 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mpnqv" Jul 23 04:39:42.202347 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:42.202108 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbzpv" Jul 23 04:39:42.202347 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:42.202260 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mpnqv" podUID="dab60874-8b3b-411d-b58e-4ec857bc9629" Jul 23 04:39:42.202347 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:42.202298 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbzpv" podUID="555665cd-6364-4d9f-8326-4973b140cc64" Jul 23 04:39:42.901817 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:42.901784 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dab60874-8b3b-411d-b58e-4ec857bc9629-metrics-certs\") pod \"network-metrics-daemon-mpnqv\" (UID: \"dab60874-8b3b-411d-b58e-4ec857bc9629\") " pod="openshift-multus/network-metrics-daemon-mpnqv" Jul 23 04:39:42.902000 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:42.901847 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/555665cd-6364-4d9f-8326-4973b140cc64-original-pull-secret\") pod \"global-pull-secret-syncer-cbzpv\" (UID: \"555665cd-6364-4d9f-8326-4973b140cc64\") " pod="kube-system/global-pull-secret-syncer-cbzpv" Jul 23 04:39:42.902000 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:42.901965 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 04:39:42.902090 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:42.902016 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 04:39:42.902090 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:42.902086 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/555665cd-6364-4d9f-8326-4973b140cc64-original-pull-secret podName:555665cd-6364-4d9f-8326-4973b140cc64 nodeName:}" failed. No retries permitted until 2026-07-23 04:39:58.902066053 +0000 UTC m=+34.244578920 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/555665cd-6364-4d9f-8326-4973b140cc64-original-pull-secret") pod "global-pull-secret-syncer-cbzpv" (UID: "555665cd-6364-4d9f-8326-4973b140cc64") : object "kube-system"/"original-pull-secret" not registered Jul 23 04:39:42.902183 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:42.902109 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dab60874-8b3b-411d-b58e-4ec857bc9629-metrics-certs podName:dab60874-8b3b-411d-b58e-4ec857bc9629 nodeName:}" failed. No retries permitted until 2026-07-23 04:39:58.902098189 +0000 UTC m=+34.244611055 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dab60874-8b3b-411d-b58e-4ec857bc9629-metrics-certs") pod "network-metrics-daemon-mpnqv" (UID: "dab60874-8b3b-411d-b58e-4ec857bc9629") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 04:39:43.002604 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:43.002572 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czjdx\" (UniqueName: \"kubernetes.io/projected/6346ee84-530c-41f1-9f8a-6d83f79daa5e-kube-api-access-czjdx\") pod \"network-check-target-7dsv7\" (UID: \"6346ee84-530c-41f1-9f8a-6d83f79daa5e\") " pod="openshift-network-diagnostics/network-check-target-7dsv7" Jul 23 04:39:43.002769 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:43.002709 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 04:39:43.002769 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:43.002728 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 04:39:43.002769 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:43.002740 2562 projected.go:196] Error preparing data for projected volume kube-api-access-czjdx for pod openshift-network-diagnostics/network-check-target-7dsv7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 04:39:43.002975 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:43.002789 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6346ee84-530c-41f1-9f8a-6d83f79daa5e-kube-api-access-czjdx podName:6346ee84-530c-41f1-9f8a-6d83f79daa5e nodeName:}" failed. No retries permitted until 2026-07-23 04:39:59.002777131 +0000 UTC m=+34.345289996 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-czjdx" (UniqueName: "kubernetes.io/projected/6346ee84-530c-41f1-9f8a-6d83f79daa5e-kube-api-access-czjdx") pod "network-check-target-7dsv7" (UID: "6346ee84-530c-41f1-9f8a-6d83f79daa5e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 04:39:43.201749 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:43.201670 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dsv7" Jul 23 04:39:43.201935 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:43.201805 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dsv7" podUID="6346ee84-530c-41f1-9f8a-6d83f79daa5e" Jul 23 04:39:44.201894 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:44.201862 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbzpv" Jul 23 04:39:44.202351 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:44.201901 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mpnqv" Jul 23 04:39:44.202351 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:44.201966 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbzpv" podUID="555665cd-6364-4d9f-8326-4973b140cc64" Jul 23 04:39:44.202351 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:44.202106 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mpnqv" podUID="dab60874-8b3b-411d-b58e-4ec857bc9629" Jul 23 04:39:45.202674 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:45.202634 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dsv7" Jul 23 04:39:45.203167 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:45.202737 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dsv7" podUID="6346ee84-530c-41f1-9f8a-6d83f79daa5e" Jul 23 04:39:46.201951 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:46.201918 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mpnqv" Jul 23 04:39:46.201951 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:46.201939 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbzpv" Jul 23 04:39:46.202185 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:46.202049 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mpnqv" podUID="dab60874-8b3b-411d-b58e-4ec857bc9629" Jul 23 04:39:46.202241 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:46.202191 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbzpv" podUID="555665cd-6364-4d9f-8326-4973b140cc64" Jul 23 04:39:47.202091 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:47.201919 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dsv7" Jul 23 04:39:47.202938 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:47.202170 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dsv7" podUID="6346ee84-530c-41f1-9f8a-6d83f79daa5e" Jul 23 04:39:47.313041 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:47.312940 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vphbq_b3007292-2534-4dc0-82a3-41d6a7d34c0c/ovn-acl-logging/0.log" Jul 23 04:39:47.313405 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:47.313382 2562 generic.go:356] "Generic (PLEG): container finished" podID="b3007292-2534-4dc0-82a3-41d6a7d34c0c" containerID="0fc76117cc6930c740616de677affabb9ba4696d955542a0a6af65e8ab67e37f" exitCode=1 Jul 23 04:39:47.313486 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:47.313461 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" event={"ID":"b3007292-2534-4dc0-82a3-41d6a7d34c0c","Type":"ContainerStarted","Data":"a5761abd407e204dd45cc56bb32c775ef622beac8a7a60d0e508513b50985228"} Jul 23 04:39:47.313560 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:47.313547 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" event={"ID":"b3007292-2534-4dc0-82a3-41d6a7d34c0c","Type":"ContainerStarted","Data":"b277834332a9115dbeaf95d6e9d6b58d9bc1cd256252ee321eb8510bba281b97"} Jul 23 04:39:47.313608 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:47.313583 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" event={"ID":"b3007292-2534-4dc0-82a3-41d6a7d34c0c","Type":"ContainerDied","Data":"0fc76117cc6930c740616de677affabb9ba4696d955542a0a6af65e8ab67e37f"} Jul 23 04:39:47.313655 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:47.313612 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" event={"ID":"b3007292-2534-4dc0-82a3-41d6a7d34c0c","Type":"ContainerStarted","Data":"098b1ce3afa5c1b0e93140b02eb4472605dbd8351ea2feb4571489d51e2ee900"} Jul 23 04:39:47.314796 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:47.314771 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-grlbc" event={"ID":"d69de93b-d93f-4f10-b618-0cdcb545197a","Type":"ContainerStarted","Data":"a3b2739817afdc88983b2762cd376f9a414b3db335d939cd817ba91890c28561"} Jul 23 04:39:47.316176 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:47.316147 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-57bmk" event={"ID":"b9a49f0e-338a-47ac-90f7-281dae5538e4","Type":"ContainerStarted","Data":"0d2c8cd32b9dc821668b7fb519a3991c4b007d1179b94f094f805da967290e42"} Jul 23 04:39:47.317515 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:47.317496 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-q2kph" event={"ID":"32fdf9c3-bed9-4d00-82f9-18e6f93dbf88","Type":"ContainerStarted","Data":"cd34110917ad0891cbe3878e2903829c60e3cf81d83d003dbceb96a6e8387338"} Jul 23 04:39:47.318846 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:47.318812 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-w7d9f" event={"ID":"0bf5e605-388c-4f50-8716-c350190d11d9","Type":"ContainerStarted","Data":"a6060b41ac985b1d18671c2e2e392844052484fc63a78b8d060a2e6ac057eeef"} Jul 23 04:39:47.320111 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:47.320091 2562 generic.go:356] "Generic (PLEG): container finished" podID="c49b7ef0-9710-41cd-9d89-145654e92fc6" containerID="2d3884f3b01d0801897fc1fd8d290ee6fe2085bc3dca058b57d1868f993eaf4d" exitCode=0 Jul 23 04:39:47.320195 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:47.320170 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zzgwf" event={"ID":"c49b7ef0-9710-41cd-9d89-145654e92fc6","Type":"ContainerDied","Data":"2d3884f3b01d0801897fc1fd8d290ee6fe2085bc3dca058b57d1868f993eaf4d"} Jul 23 04:39:47.321508 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:47.321487 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zhq6d" event={"ID":"ba17063f-a4cc-4378-afe1-6b4aad36adef","Type":"ContainerStarted","Data":"3f960afcdf44f4620278b77cadb06fbec3afcdf544a6481bbea5c6921746e942"} Jul 23 04:39:47.322778 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:47.322757 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9nzz9" event={"ID":"43514e24-f4a7-4dfc-b45d-12d28e0a4bfc","Type":"ContainerStarted","Data":"c5dba074e8e8c1a496f0d222b72f2e064bcb39d3451f6ab010c1f6218c22ebdc"} Jul 23 04:39:47.333949 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:47.333876 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-grlbc" podStartSLOduration=3.735586476 podStartE2EDuration="22.333865255s" totalImagesPullingTime="18.598278779s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 04:39:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:39:47.333672214 +0000 UTC m=+22.676185103" watchObservedRunningTime="2026-07-23 04:39:47.333865255 +0000 UTC m=+22.676378143" Jul 23 04:39:47.381093 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:47.381054 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-57bmk" podStartSLOduration=3.78387995 podStartE2EDuration="22.381027406s" totalImagesPullingTime="18.597147456s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 04:39:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:39:47.357512989 +0000 UTC m=+22.700025878" watchObservedRunningTime="2026-07-23 04:39:47.381027406 +0000 UTC m=+22.723540312" Jul 23 04:39:47.381413 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:47.381384 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-zhq6d" podStartSLOduration=3.776192372 podStartE2EDuration="22.381376606s" totalImagesPullingTime="18.605184234s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:39:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:39:47.380761179 +0000 UTC m=+22.723274066" watchObservedRunningTime="2026-07-23 04:39:47.381376606 +0000 UTC m=+22.723889495" Jul 23 04:39:47.428593 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:47.428545 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-w7d9f" podStartSLOduration=3.737542044 podStartE2EDuration="22.428527654s" totalImagesPullingTime="18.69098561s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 04:39:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:39:47.40780024 +0000 UTC m=+22.750313129" watchObservedRunningTime="2026-07-23 04:39:47.428527654 +0000 UTC m=+22.771040543" Jul 23 04:39:47.428959 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:47.428925 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-q2kph" podStartSLOduration=3.827277306 podStartE2EDuration="22.428909919s" totalImagesPullingTime="18.601632613s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 04:39:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:39:47.427216582 +0000 UTC m=+22.769729469" watchObservedRunningTime="2026-07-23 04:39:47.428909919 +0000 UTC m=+22.771422808" Jul 23 04:39:48.201728 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:48.201699 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mpnqv" Jul 23 04:39:48.202017 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:48.201824 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mpnqv" podUID="dab60874-8b3b-411d-b58e-4ec857bc9629" Jul 23 04:39:48.202017 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:48.201931 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbzpv" Jul 23 04:39:48.202106 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:48.202023 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbzpv" podUID="555665cd-6364-4d9f-8326-4973b140cc64" Jul 23 04:39:48.234478 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:48.234458 2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 23 04:39:48.328028 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:48.327996 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9nzz9" event={"ID":"43514e24-f4a7-4dfc-b45d-12d28e0a4bfc","Type":"ContainerStarted","Data":"730bf6c8f859787bc50aa138dc307fb33861ecd907f34b6b5110f2c879b72ed8"} Jul 23 04:39:48.331173 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:48.331144 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vphbq_b3007292-2534-4dc0-82a3-41d6a7d34c0c/ovn-acl-logging/0.log" Jul 23 04:39:48.331610 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:48.331581 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" event={"ID":"b3007292-2534-4dc0-82a3-41d6a7d34c0c","Type":"ContainerStarted","Data":"ec4ecca0f9b158ac831d8ba2029f098b54ceeebcb81636e74c695920bc554c07"} Jul 23 04:39:48.331731 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:48.331665 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" event={"ID":"b3007292-2534-4dc0-82a3-41d6a7d34c0c","Type":"ContainerStarted","Data":"35d38757e831012acb70745a6fe38a621f432ce0ca9d215c2375c8e00b07c73b"} Jul 23 04:39:48.333215 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:48.333192 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-htlgx" event={"ID":"67a7fb5e-8e74-4275-8abd-56f340394bd0","Type":"ContainerStarted","Data":"81419ab5b5cef83fedca93ccc705aee342cf89610fb80a5e450bcd950a4deb07"} Jul 23 04:39:49.132464 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:49.132321 2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-23T04:39:48.234473641Z","UUID":"d386ae4e-ea43-4709-ab0f-0c8cb9f2f7e3","Handler":null,"Name":"","Endpoint":""} Jul 23 04:39:49.134525 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:49.134499 2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 23 04:39:49.134646 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:49.134540 2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 23 04:39:49.202086 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:49.202053 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dsv7" Jul 23 04:39:49.202231 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:49.202207 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dsv7" podUID="6346ee84-530c-41f1-9f8a-6d83f79daa5e" Jul 23 04:39:49.338135 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:49.338102 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9nzz9" event={"ID":"43514e24-f4a7-4dfc-b45d-12d28e0a4bfc","Type":"ContainerStarted","Data":"f5dfc85f0ac81a58686daf7033810bb2e0a25ddde596ecb65a91a6a008f0be1e"} Jul 23 04:39:49.356995 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:49.356948 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9nzz9" podStartSLOduration=3.601399947 podStartE2EDuration="24.356928123s" totalImagesPullingTime="20.755528176s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 04:39:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:39:49.356735386 +0000 UTC m=+24.699248274" watchObservedRunningTime="2026-07-23 04:39:49.356928123 +0000 UTC m=+24.699441012" Jul 23 04:39:49.357468 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:49.357440 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-htlgx" podStartSLOduration=5.749631951 podStartE2EDuration="24.357429781s" totalImagesPullingTime="18.60779783s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 04:39:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:39:48.351596931 +0000 UTC m=+23.694109819" watchObservedRunningTime="2026-07-23 04:39:49.357429781 +0000 UTC m=+24.699942672" Jul 23 04:39:50.202138 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:50.202098 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mpnqv" Jul 23 04:39:50.202138 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:50.202128 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbzpv" Jul 23 04:39:50.202353 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:50.202231 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mpnqv" podUID="dab60874-8b3b-411d-b58e-4ec857bc9629" Jul 23 04:39:50.202416 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:50.202366 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbzpv" podUID="555665cd-6364-4d9f-8326-4973b140cc64" Jul 23 04:39:50.229164 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:50.229135 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-zhq6d" Jul 23 04:39:50.229825 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:50.229806 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-zhq6d" Jul 23 04:39:50.344468 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:50.344441 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vphbq_b3007292-2534-4dc0-82a3-41d6a7d34c0c/ovn-acl-logging/0.log" Jul 23 04:39:50.344959 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:50.344930 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" event={"ID":"b3007292-2534-4dc0-82a3-41d6a7d34c0c","Type":"ContainerStarted","Data":"70f2987a850996b656c98a52ce5cb46ea7d5f1bc00746574d1b81054a03646fb"} Jul 23 04:39:50.345297 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:50.345234 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-zhq6d" Jul 23 04:39:50.345573 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:50.345553 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zhq6d" Jul 23 04:39:51.202221 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:51.202025 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dsv7" Jul 23 04:39:51.202398 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:51.202327 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dsv7" podUID="6346ee84-530c-41f1-9f8a-6d83f79daa5e" Jul 23 04:39:52.202252 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:52.202072 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mpnqv" Jul 23 04:39:52.202818 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:52.202084 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbzpv" Jul 23 04:39:52.202818 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:52.202328 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mpnqv" podUID="dab60874-8b3b-411d-b58e-4ec857bc9629" Jul 23 04:39:52.202818 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:52.202386 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbzpv" podUID="555665cd-6364-4d9f-8326-4973b140cc64" Jul 23 04:39:52.350646 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:52.350617 2562 generic.go:356] "Generic (PLEG): container finished" podID="c49b7ef0-9710-41cd-9d89-145654e92fc6" containerID="5472b73b089acbab76534e4e0c69bfd62a1c779a9d12feeb0cffcfbc3c57269c" exitCode=0 Jul 23 04:39:52.350763 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:52.350696 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zzgwf" event={"ID":"c49b7ef0-9710-41cd-9d89-145654e92fc6","Type":"ContainerDied","Data":"5472b73b089acbab76534e4e0c69bfd62a1c779a9d12feeb0cffcfbc3c57269c"} Jul 23 04:39:52.354416 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:52.354401 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vphbq_b3007292-2534-4dc0-82a3-41d6a7d34c0c/ovn-acl-logging/0.log" Jul 23 04:39:52.354756 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:52.354733 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" event={"ID":"b3007292-2534-4dc0-82a3-41d6a7d34c0c","Type":"ContainerStarted","Data":"7f68c7a905a58defee388bc43f095f1304076d5c881772fc57ec0202258c2bca"} Jul 23 04:39:52.355073 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:52.355048 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:52.355215 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:52.355197 2562 scope.go:117] "RemoveContainer" containerID="0fc76117cc6930c740616de677affabb9ba4696d955542a0a6af65e8ab67e37f" Jul 23 04:39:52.370636 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:52.370618 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:53.202045 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:53.201994 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dsv7" Jul 23 04:39:53.202128 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:53.202097 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dsv7" podUID="6346ee84-530c-41f1-9f8a-6d83f79daa5e" Jul 23 04:39:53.359364 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:53.359326 2562 generic.go:356] "Generic (PLEG): container finished" podID="c49b7ef0-9710-41cd-9d89-145654e92fc6" containerID="cccb63da9a714f8abd25cb0a6a621b3e6c9ba34f2dd5bc9787d251a97bfe0ba5" exitCode=0 Jul 23 04:39:53.359766 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:53.359394 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zzgwf" event={"ID":"c49b7ef0-9710-41cd-9d89-145654e92fc6","Type":"ContainerDied","Data":"cccb63da9a714f8abd25cb0a6a621b3e6c9ba34f2dd5bc9787d251a97bfe0ba5"} Jul 23 04:39:53.363550 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:53.363533 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vphbq_b3007292-2534-4dc0-82a3-41d6a7d34c0c/ovn-acl-logging/0.log" Jul 23 04:39:53.364083 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:53.364068 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" event={"ID":"b3007292-2534-4dc0-82a3-41d6a7d34c0c","Type":"ContainerStarted","Data":"598236339a74ba3d3a3ce48779efcce3569495677d2e1c748aeb3c46541e04cb"} Jul 23 04:39:53.364259 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:53.364245 2562 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 23 04:39:53.364480 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:53.364462 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:53.379376 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:53.379358 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:53.428966 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:53.428921 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" podStartSLOduration=9.749689235 podStartE2EDuration="28.428903216s" totalImagesPullingTime="18.679213981s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 04:39:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:39:53.426973647 +0000 UTC m=+28.769486540" watchObservedRunningTime="2026-07-23 04:39:53.428903216 +0000 UTC m=+28.771416155" Jul 23 04:39:53.716286 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:53.716253 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cbzpv"] Jul 23 04:39:53.716438 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:53.716377 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbzpv" Jul 23 04:39:53.716499 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:53.716482 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbzpv" podUID="555665cd-6364-4d9f-8326-4973b140cc64" Jul 23 04:39:53.718378 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:53.718355 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7dsv7"] Jul 23 04:39:53.718507 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:53.718419 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dsv7" Jul 23 04:39:53.718507 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:53.718481 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dsv7" podUID="6346ee84-530c-41f1-9f8a-6d83f79daa5e" Jul 23 04:39:53.719789 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:53.719771 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:39:53.731392 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:53.731344 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mpnqv"] Jul 23 04:39:53.731468 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:53.731424 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mpnqv" Jul 23 04:39:53.731517 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:53.731501 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mpnqv" podUID="dab60874-8b3b-411d-b58e-4ec857bc9629" Jul 23 04:39:54.368675 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:54.368646 2562 generic.go:356] "Generic (PLEG): container finished" podID="c49b7ef0-9710-41cd-9d89-145654e92fc6" containerID="8c06d96b17f44a8537c25d21e7dd01e6db0992bae6cb18790271a2620c0e0ee9" exitCode=0 Jul 23 04:39:54.369019 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:54.368723 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zzgwf" event={"ID":"c49b7ef0-9710-41cd-9d89-145654e92fc6","Type":"ContainerDied","Data":"8c06d96b17f44a8537c25d21e7dd01e6db0992bae6cb18790271a2620c0e0ee9"} Jul 23 04:39:55.203075 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:55.203031 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mpnqv" Jul 23 04:39:55.203234 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:55.203135 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dsv7" Jul 23 04:39:55.203234 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:55.203174 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbzpv" Jul 23 04:39:55.203234 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:55.203180 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mpnqv" podUID="dab60874-8b3b-411d-b58e-4ec857bc9629" Jul 23 04:39:55.203389 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:55.203251 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbzpv" podUID="555665cd-6364-4d9f-8326-4973b140cc64" Jul 23 04:39:55.203389 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:55.203310 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dsv7" podUID="6346ee84-530c-41f1-9f8a-6d83f79daa5e" Jul 23 04:39:57.201370 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:57.201326 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mpnqv" Jul 23 04:39:57.201370 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:57.201358 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbzpv" Jul 23 04:39:57.201963 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:57.201330 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dsv7" Jul 23 04:39:57.201963 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:57.201452 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mpnqv" podUID="dab60874-8b3b-411d-b58e-4ec857bc9629" Jul 23 04:39:57.201963 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:57.201505 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dsv7" podUID="6346ee84-530c-41f1-9f8a-6d83f79daa5e" Jul 23 04:39:57.201963 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:57.201570 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbzpv" podUID="555665cd-6364-4d9f-8326-4973b140cc64" Jul 23 04:39:58.926362 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:58.926338 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dab60874-8b3b-411d-b58e-4ec857bc9629-metrics-certs\") pod \"network-metrics-daemon-mpnqv\" (UID: \"dab60874-8b3b-411d-b58e-4ec857bc9629\") " pod="openshift-multus/network-metrics-daemon-mpnqv" Jul 23 04:39:58.926673 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:58.926370 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/555665cd-6364-4d9f-8326-4973b140cc64-original-pull-secret\") pod \"global-pull-secret-syncer-cbzpv\" (UID: \"555665cd-6364-4d9f-8326-4973b140cc64\") " pod="kube-system/global-pull-secret-syncer-cbzpv" Jul 23 04:39:58.926673 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:58.926449 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 04:39:58.926673 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:58.926474 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 04:39:58.926673 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:58.926494 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/555665cd-6364-4d9f-8326-4973b140cc64-original-pull-secret podName:555665cd-6364-4d9f-8326-4973b140cc64 nodeName:}" failed. No retries permitted until 2026-07-23 04:40:30.926482049 +0000 UTC m=+66.268994916 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/555665cd-6364-4d9f-8326-4973b140cc64-original-pull-secret") pod "global-pull-secret-syncer-cbzpv" (UID: "555665cd-6364-4d9f-8326-4973b140cc64") : object "kube-system"/"original-pull-secret" not registered Jul 23 04:39:58.926673 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:58.926525 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dab60874-8b3b-411d-b58e-4ec857bc9629-metrics-certs podName:dab60874-8b3b-411d-b58e-4ec857bc9629 nodeName:}" failed. No retries permitted until 2026-07-23 04:40:30.926512661 +0000 UTC m=+66.269025527 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dab60874-8b3b-411d-b58e-4ec857bc9629-metrics-certs") pod "network-metrics-daemon-mpnqv" (UID: "dab60874-8b3b-411d-b58e-4ec857bc9629") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 04:39:59.026645 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.026617 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czjdx\" (UniqueName: \"kubernetes.io/projected/6346ee84-530c-41f1-9f8a-6d83f79daa5e-kube-api-access-czjdx\") pod \"network-check-target-7dsv7\" (UID: \"6346ee84-530c-41f1-9f8a-6d83f79daa5e\") " pod="openshift-network-diagnostics/network-check-target-7dsv7" Jul 23 04:39:59.026767 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:59.026753 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 04:39:59.026817 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:59.026771 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 04:39:59.026817 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:59.026781 2562 projected.go:196] Error preparing data for projected volume kube-api-access-czjdx for pod openshift-network-diagnostics/network-check-target-7dsv7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 04:39:59.026904 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:59.026826 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6346ee84-530c-41f1-9f8a-6d83f79daa5e-kube-api-access-czjdx podName:6346ee84-530c-41f1-9f8a-6d83f79daa5e nodeName:}" failed. No retries permitted until 2026-07-23 04:40:31.026812314 +0000 UTC m=+66.369325181 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-czjdx" (UniqueName: "kubernetes.io/projected/6346ee84-530c-41f1-9f8a-6d83f79daa5e-kube-api-access-czjdx") pod "network-check-target-7dsv7" (UID: "6346ee84-530c-41f1-9f8a-6d83f79daa5e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 04:39:59.202155 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.202089 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mpnqv" Jul 23 04:39:59.202262 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.202097 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbzpv" Jul 23 04:39:59.202262 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:59.202205 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mpnqv" podUID="dab60874-8b3b-411d-b58e-4ec857bc9629" Jul 23 04:39:59.202262 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.202105 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dsv7" Jul 23 04:39:59.202386 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:59.202267 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbzpv" podUID="555665cd-6364-4d9f-8326-4973b140cc64" Jul 23 04:39:59.202386 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:59.202321 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7dsv7" podUID="6346ee84-530c-41f1-9f8a-6d83f79daa5e" Jul 23 04:39:59.382742 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.382716 2562 generic.go:356] "Generic (PLEG): container finished" podID="c49b7ef0-9710-41cd-9d89-145654e92fc6" containerID="c46d94347eb161b6efb192b6eb93a65dfee2ac500fcf2b4a9914c02aacb3a31a" exitCode=0 Jul 23 04:39:59.382857 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.382769 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zzgwf" event={"ID":"c49b7ef0-9710-41cd-9d89-145654e92fc6","Type":"ContainerDied","Data":"c46d94347eb161b6efb192b6eb93a65dfee2ac500fcf2b4a9914c02aacb3a31a"} Jul 23 04:39:59.433385 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.433077 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-96.ec2.internal" event="NodeReady" Jul 23 04:39:59.433385 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.433210 2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 23 04:39:59.506159 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.506136 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-45c4t"] Jul 23 04:39:59.523533 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.523511 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qj5zw"] Jul 23 04:39:59.523668 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.523648 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-45c4t" Jul 23 04:39:59.536180 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.536156 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 23 04:39:59.536297 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.536197 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 23 04:39:59.536297 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.536273 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8x9dj\"" Jul 23 04:39:59.545735 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.545716 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-45c4t"] Jul 23 04:39:59.545826 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.545747 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qj5zw"] Jul 23 04:39:59.545826 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.545815 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qj5zw" Jul 23 04:39:59.550503 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.550485 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 23 04:39:59.551065 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.551042 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 23 04:39:59.552248 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.552229 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-k9hcj\"" Jul 23 04:39:59.558130 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.558113 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 23 04:39:59.631480 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.631445 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/10491e7a-c5ab-4d40-a5ff-30d5714ffdf9-config-volume\") pod \"dns-default-45c4t\" (UID: \"10491e7a-c5ab-4d40-a5ff-30d5714ffdf9\") " pod="openshift-dns/dns-default-45c4t" Jul 23 04:39:59.631480 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.631479 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8fb208f1-6626-4be0-9bfa-7cb8766d58ec-cert\") pod \"ingress-canary-qj5zw\" (UID: \"8fb208f1-6626-4be0-9bfa-7cb8766d58ec\") " pod="openshift-ingress-canary/ingress-canary-qj5zw" Jul 23 04:39:59.631645 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.631554 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/10491e7a-c5ab-4d40-a5ff-30d5714ffdf9-metrics-tls\") pod \"dns-default-45c4t\" (UID: \"10491e7a-c5ab-4d40-a5ff-30d5714ffdf9\") " pod="openshift-dns/dns-default-45c4t" Jul 23 04:39:59.631645 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.631571 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twtzj\" (UniqueName: \"kubernetes.io/projected/8fb208f1-6626-4be0-9bfa-7cb8766d58ec-kube-api-access-twtzj\") pod \"ingress-canary-qj5zw\" (UID: \"8fb208f1-6626-4be0-9bfa-7cb8766d58ec\") " pod="openshift-ingress-canary/ingress-canary-qj5zw" Jul 23 04:39:59.631645 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.631604 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10491e7a-c5ab-4d40-a5ff-30d5714ffdf9-tmp-dir\") pod \"dns-default-45c4t\" (UID: \"10491e7a-c5ab-4d40-a5ff-30d5714ffdf9\") " pod="openshift-dns/dns-default-45c4t" Jul 23 04:39:59.631645 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.631623 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sg92h\" (UniqueName: \"kubernetes.io/projected/10491e7a-c5ab-4d40-a5ff-30d5714ffdf9-kube-api-access-sg92h\") pod \"dns-default-45c4t\" (UID: \"10491e7a-c5ab-4d40-a5ff-30d5714ffdf9\") " pod="openshift-dns/dns-default-45c4t" Jul 23 04:39:59.732957 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.732879 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/10491e7a-c5ab-4d40-a5ff-30d5714ffdf9-metrics-tls\") pod \"dns-default-45c4t\" (UID: \"10491e7a-c5ab-4d40-a5ff-30d5714ffdf9\") " pod="openshift-dns/dns-default-45c4t" Jul 23 04:39:59.732957 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.732915 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twtzj\" (UniqueName: \"kubernetes.io/projected/8fb208f1-6626-4be0-9bfa-7cb8766d58ec-kube-api-access-twtzj\") pod \"ingress-canary-qj5zw\" (UID: \"8fb208f1-6626-4be0-9bfa-7cb8766d58ec\") " pod="openshift-ingress-canary/ingress-canary-qj5zw" Jul 23 04:39:59.732957 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.732938 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10491e7a-c5ab-4d40-a5ff-30d5714ffdf9-tmp-dir\") pod \"dns-default-45c4t\" (UID: \"10491e7a-c5ab-4d40-a5ff-30d5714ffdf9\") " pod="openshift-dns/dns-default-45c4t" Jul 23 04:39:59.733130 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.732996 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg92h\" (UniqueName: \"kubernetes.io/projected/10491e7a-c5ab-4d40-a5ff-30d5714ffdf9-kube-api-access-sg92h\") pod \"dns-default-45c4t\" (UID: \"10491e7a-c5ab-4d40-a5ff-30d5714ffdf9\") " pod="openshift-dns/dns-default-45c4t" Jul 23 04:39:59.733130 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.733056 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/10491e7a-c5ab-4d40-a5ff-30d5714ffdf9-config-volume\") pod \"dns-default-45c4t\" (UID: \"10491e7a-c5ab-4d40-a5ff-30d5714ffdf9\") " pod="openshift-dns/dns-default-45c4t" Jul 23 04:39:59.733130 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.733079 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8fb208f1-6626-4be0-9bfa-7cb8766d58ec-cert\") pod \"ingress-canary-qj5zw\" (UID: \"8fb208f1-6626-4be0-9bfa-7cb8766d58ec\") " pod="openshift-ingress-canary/ingress-canary-qj5zw" Jul 23 04:39:59.733130 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:59.733113 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 04:39:59.733304 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:39:59.733201 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10491e7a-c5ab-4d40-a5ff-30d5714ffdf9-metrics-tls podName:10491e7a-c5ab-4d40-a5ff-30d5714ffdf9 nodeName:}" failed. No retries permitted until 2026-07-23 04:40:00.233182477 +0000 UTC m=+35.575695343 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/10491e7a-c5ab-4d40-a5ff-30d5714ffdf9-metrics-tls") pod "dns-default-45c4t" (UID: "10491e7a-c5ab-4d40-a5ff-30d5714ffdf9") : secret "dns-default-metrics-tls" not found Jul 23 04:39:59.733304 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.733228 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10491e7a-c5ab-4d40-a5ff-30d5714ffdf9-tmp-dir\") pod \"dns-default-45c4t\" (UID: \"10491e7a-c5ab-4d40-a5ff-30d5714ffdf9\") " pod="openshift-dns/dns-default-45c4t" Jul 23 04:39:59.733568 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.733551 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/10491e7a-c5ab-4d40-a5ff-30d5714ffdf9-config-volume\") pod \"dns-default-45c4t\" (UID: \"10491e7a-c5ab-4d40-a5ff-30d5714ffdf9\") " pod="openshift-dns/dns-default-45c4t" Jul 23 04:39:59.737395 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.737376 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8fb208f1-6626-4be0-9bfa-7cb8766d58ec-cert\") pod \"ingress-canary-qj5zw\" (UID: \"8fb208f1-6626-4be0-9bfa-7cb8766d58ec\") " pod="openshift-ingress-canary/ingress-canary-qj5zw" Jul 23 04:39:59.744981 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.744959 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sg92h\" (UniqueName: \"kubernetes.io/projected/10491e7a-c5ab-4d40-a5ff-30d5714ffdf9-kube-api-access-sg92h\") pod \"dns-default-45c4t\" (UID: \"10491e7a-c5ab-4d40-a5ff-30d5714ffdf9\") " pod="openshift-dns/dns-default-45c4t" Jul 23 04:39:59.745251 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.745234 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twtzj\" (UniqueName: \"kubernetes.io/projected/8fb208f1-6626-4be0-9bfa-7cb8766d58ec-kube-api-access-twtzj\") pod \"ingress-canary-qj5zw\" (UID: \"8fb208f1-6626-4be0-9bfa-7cb8766d58ec\") " pod="openshift-ingress-canary/ingress-canary-qj5zw" Jul 23 04:39:59.853441 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:39:59.853388 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qj5zw" Jul 23 04:40:00.061385 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:00.061163 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qj5zw"] Jul 23 04:40:00.064409 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:40:00.064383 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fb208f1_6626_4be0_9bfa_7cb8766d58ec.slice/crio-06b98a78a3aabd7a4c7535b58fdd007007c77e0c4ef822244b5bb1b8e8d90e90 WatchSource:0}: Error finding container 06b98a78a3aabd7a4c7535b58fdd007007c77e0c4ef822244b5bb1b8e8d90e90: Status 404 returned error can't find the container with id 06b98a78a3aabd7a4c7535b58fdd007007c77e0c4ef822244b5bb1b8e8d90e90 Jul 23 04:40:00.237335 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:00.237302 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/10491e7a-c5ab-4d40-a5ff-30d5714ffdf9-metrics-tls\") pod \"dns-default-45c4t\" (UID: \"10491e7a-c5ab-4d40-a5ff-30d5714ffdf9\") " pod="openshift-dns/dns-default-45c4t" Jul 23 04:40:00.237489 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:00.237419 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 04:40:00.237489 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:00.237481 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10491e7a-c5ab-4d40-a5ff-30d5714ffdf9-metrics-tls podName:10491e7a-c5ab-4d40-a5ff-30d5714ffdf9 nodeName:}" failed. No retries permitted until 2026-07-23 04:40:01.237467649 +0000 UTC m=+36.579980516 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/10491e7a-c5ab-4d40-a5ff-30d5714ffdf9-metrics-tls") pod "dns-default-45c4t" (UID: "10491e7a-c5ab-4d40-a5ff-30d5714ffdf9") : secret "dns-default-metrics-tls" not found Jul 23 04:40:00.387809 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:00.387777 2562 generic.go:356] "Generic (PLEG): container finished" podID="c49b7ef0-9710-41cd-9d89-145654e92fc6" containerID="6a2ba035f4b82bdfc72f6d0654ee0563fed3182030fb479196ef66a04728d276" exitCode=0 Jul 23 04:40:00.388014 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:00.387860 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zzgwf" event={"ID":"c49b7ef0-9710-41cd-9d89-145654e92fc6","Type":"ContainerDied","Data":"6a2ba035f4b82bdfc72f6d0654ee0563fed3182030fb479196ef66a04728d276"} Jul 23 04:40:00.388969 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:00.388944 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qj5zw" event={"ID":"8fb208f1-6626-4be0-9bfa-7cb8766d58ec","Type":"ContainerStarted","Data":"06b98a78a3aabd7a4c7535b58fdd007007c77e0c4ef822244b5bb1b8e8d90e90"} Jul 23 04:40:01.201981 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.201944 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbzpv" Jul 23 04:40:01.202493 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.201944 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dsv7" Jul 23 04:40:01.202493 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.201944 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mpnqv" Jul 23 04:40:01.204805 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.204784 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 23 04:40:01.204805 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.204797 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 23 04:40:01.205006 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.204788 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 23 04:40:01.205853 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.205817 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j2njx\"" Jul 23 04:40:01.205947 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.205821 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-m89hb\"" Jul 23 04:40:01.205947 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.205827 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 23 04:40:01.246203 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.246178 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/10491e7a-c5ab-4d40-a5ff-30d5714ffdf9-metrics-tls\") pod \"dns-default-45c4t\" (UID: \"10491e7a-c5ab-4d40-a5ff-30d5714ffdf9\") " pod="openshift-dns/dns-default-45c4t" Jul 23 04:40:01.246357 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:01.246339 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 04:40:01.246420 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:01.246409 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10491e7a-c5ab-4d40-a5ff-30d5714ffdf9-metrics-tls podName:10491e7a-c5ab-4d40-a5ff-30d5714ffdf9 nodeName:}" failed. No retries permitted until 2026-07-23 04:40:03.246393431 +0000 UTC m=+38.588906297 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/10491e7a-c5ab-4d40-a5ff-30d5714ffdf9-metrics-tls") pod "dns-default-45c4t" (UID: "10491e7a-c5ab-4d40-a5ff-30d5714ffdf9") : secret "dns-default-metrics-tls" not found Jul 23 04:40:01.290242 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.290216 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4p4x8"] Jul 23 04:40:01.309490 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.309470 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4p4x8"] Jul 23 04:40:01.309601 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.309556 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4p4x8" Jul 23 04:40:01.312253 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.312065 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 23 04:40:01.312348 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.312274 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 23 04:40:01.312568 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.312551 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-hkpns\"" Jul 23 04:40:01.396124 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.396087 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zzgwf" event={"ID":"c49b7ef0-9710-41cd-9d89-145654e92fc6","Type":"ContainerStarted","Data":"b8c2d74c7c230a62489f1cee4d7273a013e43909b865120cc2ee3d4b03778f79"} Jul 23 04:40:01.398312 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.398291 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-8v8xp"] Jul 23 04:40:01.413284 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.413255 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-8v8xp" Jul 23 04:40:01.413754 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.413633 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-8v8xp"] Jul 23 04:40:01.416254 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.416234 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 23 04:40:01.416354 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.416315 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 23 04:40:01.416415 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.416387 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-j4xlt\"" Jul 23 04:40:01.416467 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.416419 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 23 04:40:01.416576 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.416562 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 23 04:40:01.422766 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.422745 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 23 04:40:01.422971 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.422935 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zzgwf" podStartSLOduration=7.306558238 podStartE2EDuration="36.422922037s" totalImagesPullingTime="29.116363799s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 04:39:25 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:40:01.422557802 +0000 UTC m=+36.765070689" watchObservedRunningTime="2026-07-23 04:40:01.422922037 +0000 UTC m=+36.765434924" Jul 23 04:40:01.448313 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.448288 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpnq6\" (UniqueName: \"kubernetes.io/projected/d8777622-488c-4307-a8fc-5d4af6137595-kube-api-access-bpnq6\") pod \"volume-data-source-validator-78cf4c6f6-4p4x8\" (UID: \"d8777622-488c-4307-a8fc-5d4af6137595\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4p4x8" Jul 23 04:40:01.499751 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.499684 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6798b7cd97-rbnww"] Jul 23 04:40:01.519326 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.519297 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6798b7cd97-rbnww"] Jul 23 04:40:01.519448 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.519386 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" Jul 23 04:40:01.521929 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.521904 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 23 04:40:01.521929 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.521935 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-ghfkp\"" Jul 23 04:40:01.522083 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.521958 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 23 04:40:01.522083 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.522046 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 23 04:40:01.529136 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.529112 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 23 04:40:01.548647 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.548622 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpnq6\" (UniqueName: \"kubernetes.io/projected/d8777622-488c-4307-a8fc-5d4af6137595-kube-api-access-bpnq6\") pod \"volume-data-source-validator-78cf4c6f6-4p4x8\" (UID: \"d8777622-488c-4307-a8fc-5d4af6137595\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4p4x8" Jul 23 04:40:01.548768 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.548652 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pnjg\" (UniqueName: \"kubernetes.io/projected/aa0558ac-8329-4a03-8a88-76a2a42d49a1-kube-api-access-5pnjg\") pod \"insights-operator-555d9d7885-8v8xp\" (UID: \"aa0558ac-8329-4a03-8a88-76a2a42d49a1\") " pod="openshift-insights/insights-operator-555d9d7885-8v8xp" Jul 23 04:40:01.548768 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.548673 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa0558ac-8329-4a03-8a88-76a2a42d49a1-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-8v8xp\" (UID: \"aa0558ac-8329-4a03-8a88-76a2a42d49a1\") " pod="openshift-insights/insights-operator-555d9d7885-8v8xp" Jul 23 04:40:01.548768 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.548695 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/aa0558ac-8329-4a03-8a88-76a2a42d49a1-snapshots\") pod \"insights-operator-555d9d7885-8v8xp\" (UID: \"aa0558ac-8329-4a03-8a88-76a2a42d49a1\") " pod="openshift-insights/insights-operator-555d9d7885-8v8xp" Jul 23 04:40:01.548971 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.548766 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa0558ac-8329-4a03-8a88-76a2a42d49a1-tmp\") pod \"insights-operator-555d9d7885-8v8xp\" (UID: \"aa0558ac-8329-4a03-8a88-76a2a42d49a1\") " pod="openshift-insights/insights-operator-555d9d7885-8v8xp" Jul 23 04:40:01.548971 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.548797 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa0558ac-8329-4a03-8a88-76a2a42d49a1-service-ca-bundle\") pod \"insights-operator-555d9d7885-8v8xp\" (UID: \"aa0558ac-8329-4a03-8a88-76a2a42d49a1\") " pod="openshift-insights/insights-operator-555d9d7885-8v8xp" Jul 23 04:40:01.548971 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.548877 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/aa0558ac-8329-4a03-8a88-76a2a42d49a1-serving-cert\") pod \"insights-operator-555d9d7885-8v8xp\" (UID: \"aa0558ac-8329-4a03-8a88-76a2a42d49a1\") " pod="openshift-insights/insights-operator-555d9d7885-8v8xp" Jul 23 04:40:01.559141 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.559118 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpnq6\" (UniqueName: \"kubernetes.io/projected/d8777622-488c-4307-a8fc-5d4af6137595-kube-api-access-bpnq6\") pod \"volume-data-source-validator-78cf4c6f6-4p4x8\" (UID: \"d8777622-488c-4307-a8fc-5d4af6137595\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4p4x8" Jul 23 04:40:01.607734 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.607701 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-657f4d9d48-7l4rd"] Jul 23 04:40:01.619934 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.619903 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4p4x8" Jul 23 04:40:01.633823 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.633802 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-657f4d9d48-7l4rd"] Jul 23 04:40:01.633956 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.633940 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-657f4d9d48-7l4rd" Jul 23 04:40:01.636857 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.636609 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 23 04:40:01.636857 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.636629 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-vfqhj\"" Jul 23 04:40:01.636857 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.636753 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 23 04:40:01.636857 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.636848 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 23 04:40:01.637108 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.636932 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 23 04:40:01.637356 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.637337 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 23 04:40:01.637681 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.637639 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 23 04:40:01.649379 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.649357 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d15db39a-c63b-4806-86a1-ba472f599e54-bound-sa-token\") pod \"image-registry-6798b7cd97-rbnww\" (UID: \"d15db39a-c63b-4806-86a1-ba472f599e54\") " pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" Jul 23 04:40:01.649460 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.649395 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa0558ac-8329-4a03-8a88-76a2a42d49a1-tmp\") pod \"insights-operator-555d9d7885-8v8xp\" (UID: \"aa0558ac-8329-4a03-8a88-76a2a42d49a1\") " pod="openshift-insights/insights-operator-555d9d7885-8v8xp" Jul 23 04:40:01.649460 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.649420 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa0558ac-8329-4a03-8a88-76a2a42d49a1-service-ca-bundle\") pod \"insights-operator-555d9d7885-8v8xp\" (UID: \"aa0558ac-8329-4a03-8a88-76a2a42d49a1\") " pod="openshift-insights/insights-operator-555d9d7885-8v8xp" Jul 23 04:40:01.649460 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.649451 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfk59\" (UniqueName: \"kubernetes.io/projected/d15db39a-c63b-4806-86a1-ba472f599e54-kube-api-access-vfk59\") pod \"image-registry-6798b7cd97-rbnww\" (UID: \"d15db39a-c63b-4806-86a1-ba472f599e54\") " pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" Jul 23 04:40:01.649616 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.649483 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d15db39a-c63b-4806-86a1-ba472f599e54-ca-trust-extracted\") pod \"image-registry-6798b7cd97-rbnww\" (UID: \"d15db39a-c63b-4806-86a1-ba472f599e54\") " pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" Jul 23 04:40:01.649616 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.649509 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/aa0558ac-8329-4a03-8a88-76a2a42d49a1-serving-cert\") pod \"insights-operator-555d9d7885-8v8xp\" (UID: \"aa0558ac-8329-4a03-8a88-76a2a42d49a1\") " pod="openshift-insights/insights-operator-555d9d7885-8v8xp" Jul 23 04:40:01.649714 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.649617 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pnjg\" (UniqueName: \"kubernetes.io/projected/aa0558ac-8329-4a03-8a88-76a2a42d49a1-kube-api-access-5pnjg\") pod \"insights-operator-555d9d7885-8v8xp\" (UID: \"aa0558ac-8329-4a03-8a88-76a2a42d49a1\") " pod="openshift-insights/insights-operator-555d9d7885-8v8xp" Jul 23 04:40:01.649714 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.649653 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa0558ac-8329-4a03-8a88-76a2a42d49a1-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-8v8xp\" (UID: \"aa0558ac-8329-4a03-8a88-76a2a42d49a1\") " pod="openshift-insights/insights-operator-555d9d7885-8v8xp" Jul 23 04:40:01.649714 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.649685 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d15db39a-c63b-4806-86a1-ba472f599e54-installation-pull-secrets\") pod \"image-registry-6798b7cd97-rbnww\" (UID: \"d15db39a-c63b-4806-86a1-ba472f599e54\") " pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" Jul 23 04:40:01.649872 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.649718 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d15db39a-c63b-4806-86a1-ba472f599e54-trusted-ca\") pod \"image-registry-6798b7cd97-rbnww\" (UID: \"d15db39a-c63b-4806-86a1-ba472f599e54\") " pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" Jul 23 04:40:01.649872 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.649750 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/aa0558ac-8329-4a03-8a88-76a2a42d49a1-snapshots\") pod \"insights-operator-555d9d7885-8v8xp\" (UID: \"aa0558ac-8329-4a03-8a88-76a2a42d49a1\") " pod="openshift-insights/insights-operator-555d9d7885-8v8xp" Jul 23 04:40:01.649872 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.649788 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d15db39a-c63b-4806-86a1-ba472f599e54-registry-tls\") pod \"image-registry-6798b7cd97-rbnww\" (UID: \"d15db39a-c63b-4806-86a1-ba472f599e54\") " pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" Jul 23 04:40:01.649872 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.649791 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa0558ac-8329-4a03-8a88-76a2a42d49a1-tmp\") pod \"insights-operator-555d9d7885-8v8xp\" (UID: \"aa0558ac-8329-4a03-8a88-76a2a42d49a1\") " pod="openshift-insights/insights-operator-555d9d7885-8v8xp" Jul 23 04:40:01.649872 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.649849 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d15db39a-c63b-4806-86a1-ba472f599e54-image-registry-private-configuration\") pod \"image-registry-6798b7cd97-rbnww\" (UID: \"d15db39a-c63b-4806-86a1-ba472f599e54\") " pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" Jul 23 04:40:01.650098 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.650018 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d15db39a-c63b-4806-86a1-ba472f599e54-registry-certificates\") pod \"image-registry-6798b7cd97-rbnww\" (UID: \"d15db39a-c63b-4806-86a1-ba472f599e54\") " pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" Jul 23 04:40:01.650098 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.650062 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa0558ac-8329-4a03-8a88-76a2a42d49a1-service-ca-bundle\") pod \"insights-operator-555d9d7885-8v8xp\" (UID: \"aa0558ac-8329-4a03-8a88-76a2a42d49a1\") " pod="openshift-insights/insights-operator-555d9d7885-8v8xp" Jul 23 04:40:01.650531 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.650513 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa0558ac-8329-4a03-8a88-76a2a42d49a1-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-8v8xp\" (UID: \"aa0558ac-8329-4a03-8a88-76a2a42d49a1\") " pod="openshift-insights/insights-operator-555d9d7885-8v8xp" Jul 23 04:40:01.650718 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.650698 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/aa0558ac-8329-4a03-8a88-76a2a42d49a1-snapshots\") pod \"insights-operator-555d9d7885-8v8xp\" (UID: \"aa0558ac-8329-4a03-8a88-76a2a42d49a1\") " pod="openshift-insights/insights-operator-555d9d7885-8v8xp" Jul 23 04:40:01.652912 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.652335 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/aa0558ac-8329-4a03-8a88-76a2a42d49a1-serving-cert\") pod \"insights-operator-555d9d7885-8v8xp\" (UID: \"aa0558ac-8329-4a03-8a88-76a2a42d49a1\") " pod="openshift-insights/insights-operator-555d9d7885-8v8xp" Jul 23 04:40:01.659289 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.659264 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pnjg\" (UniqueName: \"kubernetes.io/projected/aa0558ac-8329-4a03-8a88-76a2a42d49a1-kube-api-access-5pnjg\") pod \"insights-operator-555d9d7885-8v8xp\" (UID: \"aa0558ac-8329-4a03-8a88-76a2a42d49a1\") " pod="openshift-insights/insights-operator-555d9d7885-8v8xp" Jul 23 04:40:01.723092 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.723065 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-8v8xp" Jul 23 04:40:01.750368 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.750298 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/244b6204-6371-4924-815e-fd84477ce28d-stats-auth\") pod \"router-default-657f4d9d48-7l4rd\" (UID: \"244b6204-6371-4924-815e-fd84477ce28d\") " pod="openshift-ingress/router-default-657f4d9d48-7l4rd" Jul 23 04:40:01.750368 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.750339 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d15db39a-c63b-4806-86a1-ba472f599e54-installation-pull-secrets\") pod \"image-registry-6798b7cd97-rbnww\" (UID: \"d15db39a-c63b-4806-86a1-ba472f599e54\") " pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" Jul 23 04:40:01.750539 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.750377 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d15db39a-c63b-4806-86a1-ba472f599e54-trusted-ca\") pod \"image-registry-6798b7cd97-rbnww\" (UID: \"d15db39a-c63b-4806-86a1-ba472f599e54\") " pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" Jul 23 04:40:01.750539 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.750404 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d15db39a-c63b-4806-86a1-ba472f599e54-registry-tls\") pod \"image-registry-6798b7cd97-rbnww\" (UID: \"d15db39a-c63b-4806-86a1-ba472f599e54\") " pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" Jul 23 04:40:01.750539 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.750424 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/244b6204-6371-4924-815e-fd84477ce28d-default-certificate\") pod \"router-default-657f4d9d48-7l4rd\" (UID: \"244b6204-6371-4924-815e-fd84477ce28d\") " pod="openshift-ingress/router-default-657f4d9d48-7l4rd" Jul 23 04:40:01.750539 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.750448 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d15db39a-c63b-4806-86a1-ba472f599e54-image-registry-private-configuration\") pod \"image-registry-6798b7cd97-rbnww\" (UID: \"d15db39a-c63b-4806-86a1-ba472f599e54\") " pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" Jul 23 04:40:01.750539 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.750472 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d15db39a-c63b-4806-86a1-ba472f599e54-registry-certificates\") pod \"image-registry-6798b7cd97-rbnww\" (UID: \"d15db39a-c63b-4806-86a1-ba472f599e54\") " pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" Jul 23 04:40:01.750539 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.750497 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d15db39a-c63b-4806-86a1-ba472f599e54-bound-sa-token\") pod \"image-registry-6798b7cd97-rbnww\" (UID: \"d15db39a-c63b-4806-86a1-ba472f599e54\") " pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" Jul 23 04:40:01.750817 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:01.750545 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 04:40:01.750817 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:01.750565 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6798b7cd97-rbnww: secret "image-registry-tls" not found Jul 23 04:40:01.750817 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.750530 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfk59\" (UniqueName: \"kubernetes.io/projected/d15db39a-c63b-4806-86a1-ba472f599e54-kube-api-access-vfk59\") pod \"image-registry-6798b7cd97-rbnww\" (UID: \"d15db39a-c63b-4806-86a1-ba472f599e54\") " pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" Jul 23 04:40:01.750817 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.750606 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d15db39a-c63b-4806-86a1-ba472f599e54-ca-trust-extracted\") pod \"image-registry-6798b7cd97-rbnww\" (UID: \"d15db39a-c63b-4806-86a1-ba472f599e54\") " pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" Jul 23 04:40:01.750817 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:01.750629 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d15db39a-c63b-4806-86a1-ba472f599e54-registry-tls podName:d15db39a-c63b-4806-86a1-ba472f599e54 nodeName:}" failed. No retries permitted until 2026-07-23 04:40:02.250607682 +0000 UTC m=+37.593120565 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d15db39a-c63b-4806-86a1-ba472f599e54-registry-tls") pod "image-registry-6798b7cd97-rbnww" (UID: "d15db39a-c63b-4806-86a1-ba472f599e54") : secret "image-registry-tls" not found Jul 23 04:40:01.750817 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.750663 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qsq7\" (UniqueName: \"kubernetes.io/projected/244b6204-6371-4924-815e-fd84477ce28d-kube-api-access-4qsq7\") pod \"router-default-657f4d9d48-7l4rd\" (UID: \"244b6204-6371-4924-815e-fd84477ce28d\") " pod="openshift-ingress/router-default-657f4d9d48-7l4rd" Jul 23 04:40:01.750817 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.750700 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/244b6204-6371-4924-815e-fd84477ce28d-service-ca-bundle\") pod \"router-default-657f4d9d48-7l4rd\" (UID: \"244b6204-6371-4924-815e-fd84477ce28d\") " pod="openshift-ingress/router-default-657f4d9d48-7l4rd" Jul 23 04:40:01.750817 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.750727 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/244b6204-6371-4924-815e-fd84477ce28d-metrics-certs\") pod \"router-default-657f4d9d48-7l4rd\" (UID: \"244b6204-6371-4924-815e-fd84477ce28d\") " pod="openshift-ingress/router-default-657f4d9d48-7l4rd" Jul 23 04:40:01.751221 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.751066 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d15db39a-c63b-4806-86a1-ba472f599e54-registry-certificates\") pod \"image-registry-6798b7cd97-rbnww\" (UID: \"d15db39a-c63b-4806-86a1-ba472f599e54\") " pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" Jul 23 04:40:01.751221 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.751106 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d15db39a-c63b-4806-86a1-ba472f599e54-ca-trust-extracted\") pod \"image-registry-6798b7cd97-rbnww\" (UID: \"d15db39a-c63b-4806-86a1-ba472f599e54\") " pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" Jul 23 04:40:01.751430 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.751407 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d15db39a-c63b-4806-86a1-ba472f599e54-trusted-ca\") pod \"image-registry-6798b7cd97-rbnww\" (UID: \"d15db39a-c63b-4806-86a1-ba472f599e54\") " pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" Jul 23 04:40:01.753505 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.753482 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d15db39a-c63b-4806-86a1-ba472f599e54-image-registry-private-configuration\") pod \"image-registry-6798b7cd97-rbnww\" (UID: \"d15db39a-c63b-4806-86a1-ba472f599e54\") " pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" Jul 23 04:40:01.753621 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.753605 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d15db39a-c63b-4806-86a1-ba472f599e54-installation-pull-secrets\") pod \"image-registry-6798b7cd97-rbnww\" (UID: \"d15db39a-c63b-4806-86a1-ba472f599e54\") " pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" Jul 23 04:40:01.761792 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.761773 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d15db39a-c63b-4806-86a1-ba472f599e54-bound-sa-token\") pod \"image-registry-6798b7cd97-rbnww\" (UID: \"d15db39a-c63b-4806-86a1-ba472f599e54\") " pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" Jul 23 04:40:01.762113 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.762092 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfk59\" (UniqueName: \"kubernetes.io/projected/d15db39a-c63b-4806-86a1-ba472f599e54-kube-api-access-vfk59\") pod \"image-registry-6798b7cd97-rbnww\" (UID: \"d15db39a-c63b-4806-86a1-ba472f599e54\") " pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" Jul 23 04:40:01.851723 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.851679 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qsq7\" (UniqueName: \"kubernetes.io/projected/244b6204-6371-4924-815e-fd84477ce28d-kube-api-access-4qsq7\") pod \"router-default-657f4d9d48-7l4rd\" (UID: \"244b6204-6371-4924-815e-fd84477ce28d\") " pod="openshift-ingress/router-default-657f4d9d48-7l4rd" Jul 23 04:40:01.851926 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.851730 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/244b6204-6371-4924-815e-fd84477ce28d-service-ca-bundle\") pod \"router-default-657f4d9d48-7l4rd\" (UID: \"244b6204-6371-4924-815e-fd84477ce28d\") " pod="openshift-ingress/router-default-657f4d9d48-7l4rd" Jul 23 04:40:01.851926 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.851759 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/244b6204-6371-4924-815e-fd84477ce28d-metrics-certs\") pod \"router-default-657f4d9d48-7l4rd\" (UID: \"244b6204-6371-4924-815e-fd84477ce28d\") " pod="openshift-ingress/router-default-657f4d9d48-7l4rd" Jul 23 04:40:01.851926 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.851824 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/244b6204-6371-4924-815e-fd84477ce28d-stats-auth\") pod \"router-default-657f4d9d48-7l4rd\" (UID: \"244b6204-6371-4924-815e-fd84477ce28d\") " pod="openshift-ingress/router-default-657f4d9d48-7l4rd" Jul 23 04:40:01.851926 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:01.851908 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/244b6204-6371-4924-815e-fd84477ce28d-service-ca-bundle podName:244b6204-6371-4924-815e-fd84477ce28d nodeName:}" failed. No retries permitted until 2026-07-23 04:40:02.351885985 +0000 UTC m=+37.694398857 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/244b6204-6371-4924-815e-fd84477ce28d-service-ca-bundle") pod "router-default-657f4d9d48-7l4rd" (UID: "244b6204-6371-4924-815e-fd84477ce28d") : configmap references non-existent config key: service-ca.crt Jul 23 04:40:01.852157 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:01.851951 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 04:40:01.852157 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.851964 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/244b6204-6371-4924-815e-fd84477ce28d-default-certificate\") pod \"router-default-657f4d9d48-7l4rd\" (UID: \"244b6204-6371-4924-815e-fd84477ce28d\") " pod="openshift-ingress/router-default-657f4d9d48-7l4rd" Jul 23 04:40:01.852157 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:01.852005 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/244b6204-6371-4924-815e-fd84477ce28d-metrics-certs podName:244b6204-6371-4924-815e-fd84477ce28d nodeName:}" failed. No retries permitted until 2026-07-23 04:40:02.351988273 +0000 UTC m=+37.694501173 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/244b6204-6371-4924-815e-fd84477ce28d-metrics-certs") pod "router-default-657f4d9d48-7l4rd" (UID: "244b6204-6371-4924-815e-fd84477ce28d") : secret "router-metrics-certs-default" not found Jul 23 04:40:01.854741 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.854718 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/244b6204-6371-4924-815e-fd84477ce28d-stats-auth\") pod \"router-default-657f4d9d48-7l4rd\" (UID: \"244b6204-6371-4924-815e-fd84477ce28d\") " pod="openshift-ingress/router-default-657f4d9d48-7l4rd" Jul 23 04:40:01.855170 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.855152 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/244b6204-6371-4924-815e-fd84477ce28d-default-certificate\") pod \"router-default-657f4d9d48-7l4rd\" (UID: \"244b6204-6371-4924-815e-fd84477ce28d\") " pod="openshift-ingress/router-default-657f4d9d48-7l4rd" Jul 23 04:40:01.861686 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:01.861665 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qsq7\" (UniqueName: \"kubernetes.io/projected/244b6204-6371-4924-815e-fd84477ce28d-kube-api-access-4qsq7\") pod \"router-default-657f4d9d48-7l4rd\" (UID: \"244b6204-6371-4924-815e-fd84477ce28d\") " pod="openshift-ingress/router-default-657f4d9d48-7l4rd" Jul 23 04:40:02.256365 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:02.256309 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d15db39a-c63b-4806-86a1-ba472f599e54-registry-tls\") pod \"image-registry-6798b7cd97-rbnww\" (UID: \"d15db39a-c63b-4806-86a1-ba472f599e54\") " pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" Jul 23 04:40:02.256816 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:02.256503 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 04:40:02.256816 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:02.256522 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6798b7cd97-rbnww: secret "image-registry-tls" not found Jul 23 04:40:02.256816 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:02.256605 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d15db39a-c63b-4806-86a1-ba472f599e54-registry-tls podName:d15db39a-c63b-4806-86a1-ba472f599e54 nodeName:}" failed. No retries permitted until 2026-07-23 04:40:03.256575929 +0000 UTC m=+38.599088814 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d15db39a-c63b-4806-86a1-ba472f599e54-registry-tls") pod "image-registry-6798b7cd97-rbnww" (UID: "d15db39a-c63b-4806-86a1-ba472f599e54") : secret "image-registry-tls" not found Jul 23 04:40:02.357073 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:02.357033 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/244b6204-6371-4924-815e-fd84477ce28d-service-ca-bundle\") pod \"router-default-657f4d9d48-7l4rd\" (UID: \"244b6204-6371-4924-815e-fd84477ce28d\") " pod="openshift-ingress/router-default-657f4d9d48-7l4rd" Jul 23 04:40:02.357073 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:02.357079 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/244b6204-6371-4924-815e-fd84477ce28d-metrics-certs\") pod \"router-default-657f4d9d48-7l4rd\" (UID: \"244b6204-6371-4924-815e-fd84477ce28d\") " pod="openshift-ingress/router-default-657f4d9d48-7l4rd" Jul 23 04:40:02.357303 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:02.357195 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 04:40:02.357303 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:02.357208 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/244b6204-6371-4924-815e-fd84477ce28d-service-ca-bundle podName:244b6204-6371-4924-815e-fd84477ce28d nodeName:}" failed. No retries permitted until 2026-07-23 04:40:03.357185713 +0000 UTC m=+38.699698588 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/244b6204-6371-4924-815e-fd84477ce28d-service-ca-bundle") pod "router-default-657f4d9d48-7l4rd" (UID: "244b6204-6371-4924-815e-fd84477ce28d") : configmap references non-existent config key: service-ca.crt Jul 23 04:40:02.357303 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:02.357244 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/244b6204-6371-4924-815e-fd84477ce28d-metrics-certs podName:244b6204-6371-4924-815e-fd84477ce28d nodeName:}" failed. No retries permitted until 2026-07-23 04:40:03.357228564 +0000 UTC m=+38.699741457 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/244b6204-6371-4924-815e-fd84477ce28d-metrics-certs") pod "router-default-657f4d9d48-7l4rd" (UID: "244b6204-6371-4924-815e-fd84477ce28d") : secret "router-metrics-certs-default" not found Jul 23 04:40:02.492646 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:02.492415 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4p4x8"] Jul 23 04:40:02.497012 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:40:02.496983 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8777622_488c_4307_a8fc_5d4af6137595.slice/crio-08ae9e484e622ebc03f175f6475e60612e90c5798f0727572a5dd166710c08ad WatchSource:0}: Error finding container 08ae9e484e622ebc03f175f6475e60612e90c5798f0727572a5dd166710c08ad: Status 404 returned error can't find the container with id 08ae9e484e622ebc03f175f6475e60612e90c5798f0727572a5dd166710c08ad Jul 23 04:40:02.514205 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:02.514187 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-8v8xp"] Jul 23 04:40:02.608920 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:40:02.608893 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa0558ac_8329_4a03_8a88_76a2a42d49a1.slice/crio-3d6fed17324b94e526cfcdde69767b1a299486054912fe05383af11b0f2c2002 WatchSource:0}: Error finding container 3d6fed17324b94e526cfcdde69767b1a299486054912fe05383af11b0f2c2002: Status 404 returned error can't find the container with id 3d6fed17324b94e526cfcdde69767b1a299486054912fe05383af11b0f2c2002 Jul 23 04:40:03.265994 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:03.265962 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/10491e7a-c5ab-4d40-a5ff-30d5714ffdf9-metrics-tls\") pod \"dns-default-45c4t\" (UID: \"10491e7a-c5ab-4d40-a5ff-30d5714ffdf9\") " pod="openshift-dns/dns-default-45c4t" Jul 23 04:40:03.266736 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:03.266026 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d15db39a-c63b-4806-86a1-ba472f599e54-registry-tls\") pod \"image-registry-6798b7cd97-rbnww\" (UID: \"d15db39a-c63b-4806-86a1-ba472f599e54\") " pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" Jul 23 04:40:03.266736 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:03.266139 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 04:40:03.266736 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:03.266186 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 04:40:03.266736 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:03.266207 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6798b7cd97-rbnww: secret "image-registry-tls" not found Jul 23 04:40:03.266736 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:03.266218 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10491e7a-c5ab-4d40-a5ff-30d5714ffdf9-metrics-tls podName:10491e7a-c5ab-4d40-a5ff-30d5714ffdf9 nodeName:}" failed. No retries permitted until 2026-07-23 04:40:07.266196586 +0000 UTC m=+42.608709469 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/10491e7a-c5ab-4d40-a5ff-30d5714ffdf9-metrics-tls") pod "dns-default-45c4t" (UID: "10491e7a-c5ab-4d40-a5ff-30d5714ffdf9") : secret "dns-default-metrics-tls" not found Jul 23 04:40:03.266736 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:03.266257 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d15db39a-c63b-4806-86a1-ba472f599e54-registry-tls podName:d15db39a-c63b-4806-86a1-ba472f599e54 nodeName:}" failed. No retries permitted until 2026-07-23 04:40:05.266240101 +0000 UTC m=+40.608752979 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d15db39a-c63b-4806-86a1-ba472f599e54-registry-tls") pod "image-registry-6798b7cd97-rbnww" (UID: "d15db39a-c63b-4806-86a1-ba472f599e54") : secret "image-registry-tls" not found Jul 23 04:40:03.367531 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:03.367495 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/244b6204-6371-4924-815e-fd84477ce28d-service-ca-bundle\") pod \"router-default-657f4d9d48-7l4rd\" (UID: \"244b6204-6371-4924-815e-fd84477ce28d\") " pod="openshift-ingress/router-default-657f4d9d48-7l4rd" Jul 23 04:40:03.367697 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:03.367548 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/244b6204-6371-4924-815e-fd84477ce28d-metrics-certs\") pod \"router-default-657f4d9d48-7l4rd\" (UID: \"244b6204-6371-4924-815e-fd84477ce28d\") " pod="openshift-ingress/router-default-657f4d9d48-7l4rd" Jul 23 04:40:03.367697 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:03.367626 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/244b6204-6371-4924-815e-fd84477ce28d-service-ca-bundle podName:244b6204-6371-4924-815e-fd84477ce28d nodeName:}" failed. No retries permitted until 2026-07-23 04:40:05.367607159 +0000 UTC m=+40.710120048 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/244b6204-6371-4924-815e-fd84477ce28d-service-ca-bundle") pod "router-default-657f4d9d48-7l4rd" (UID: "244b6204-6371-4924-815e-fd84477ce28d") : configmap references non-existent config key: service-ca.crt Jul 23 04:40:03.367697 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:03.367663 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 04:40:03.367697 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:03.367698 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/244b6204-6371-4924-815e-fd84477ce28d-metrics-certs podName:244b6204-6371-4924-815e-fd84477ce28d nodeName:}" failed. No retries permitted until 2026-07-23 04:40:05.367687975 +0000 UTC m=+40.710200846 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/244b6204-6371-4924-815e-fd84477ce28d-metrics-certs") pod "router-default-657f4d9d48-7l4rd" (UID: "244b6204-6371-4924-815e-fd84477ce28d") : secret "router-metrics-certs-default" not found Jul 23 04:40:03.406610 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:03.406575 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qj5zw" event={"ID":"8fb208f1-6626-4be0-9bfa-7cb8766d58ec","Type":"ContainerStarted","Data":"2582a39343698464cc8a454a49efb3fadb89af83f9484889c44a2f32c2d7a9c2"} Jul 23 04:40:03.408084 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:03.408054 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4p4x8" event={"ID":"d8777622-488c-4307-a8fc-5d4af6137595","Type":"ContainerStarted","Data":"08ae9e484e622ebc03f175f6475e60612e90c5798f0727572a5dd166710c08ad"} Jul 23 04:40:03.409430 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:03.409403 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-8v8xp" event={"ID":"aa0558ac-8329-4a03-8a88-76a2a42d49a1","Type":"ContainerStarted","Data":"3d6fed17324b94e526cfcdde69767b1a299486054912fe05383af11b0f2c2002"} Jul 23 04:40:03.424369 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:03.424320 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qj5zw" podStartSLOduration=1.866586936 podStartE2EDuration="4.424303026s" totalImagesPullingTime="2.55771609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:39:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:40:03.423124027 +0000 UTC m=+38.765636916" watchObservedRunningTime="2026-07-23 04:40:03.424303026 +0000 UTC m=+38.766815911" Jul 23 04:40:05.287131 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:05.287098 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d15db39a-c63b-4806-86a1-ba472f599e54-registry-tls\") pod \"image-registry-6798b7cd97-rbnww\" (UID: \"d15db39a-c63b-4806-86a1-ba472f599e54\") " pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" Jul 23 04:40:05.287421 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:05.287234 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 04:40:05.287421 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:05.287250 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6798b7cd97-rbnww: secret "image-registry-tls" not found Jul 23 04:40:05.287421 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:05.287300 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d15db39a-c63b-4806-86a1-ba472f599e54-registry-tls podName:d15db39a-c63b-4806-86a1-ba472f599e54 nodeName:}" failed. No retries permitted until 2026-07-23 04:40:09.287285363 +0000 UTC m=+44.629798228 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d15db39a-c63b-4806-86a1-ba472f599e54-registry-tls") pod "image-registry-6798b7cd97-rbnww" (UID: "d15db39a-c63b-4806-86a1-ba472f599e54") : secret "image-registry-tls" not found Jul 23 04:40:05.387751 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:05.387713 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/244b6204-6371-4924-815e-fd84477ce28d-service-ca-bundle\") pod \"router-default-657f4d9d48-7l4rd\" (UID: \"244b6204-6371-4924-815e-fd84477ce28d\") " pod="openshift-ingress/router-default-657f4d9d48-7l4rd" Jul 23 04:40:05.387751 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:05.387751 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/244b6204-6371-4924-815e-fd84477ce28d-metrics-certs\") pod \"router-default-657f4d9d48-7l4rd\" (UID: \"244b6204-6371-4924-815e-fd84477ce28d\") " pod="openshift-ingress/router-default-657f4d9d48-7l4rd" Jul 23 04:40:05.387961 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:05.387889 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 04:40:05.387961 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:05.387896 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/244b6204-6371-4924-815e-fd84477ce28d-service-ca-bundle podName:244b6204-6371-4924-815e-fd84477ce28d nodeName:}" failed. No retries permitted until 2026-07-23 04:40:09.387878336 +0000 UTC m=+44.730391201 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/244b6204-6371-4924-815e-fd84477ce28d-service-ca-bundle") pod "router-default-657f4d9d48-7l4rd" (UID: "244b6204-6371-4924-815e-fd84477ce28d") : configmap references non-existent config key: service-ca.crt Jul 23 04:40:05.387961 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:05.387928 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/244b6204-6371-4924-815e-fd84477ce28d-metrics-certs podName:244b6204-6371-4924-815e-fd84477ce28d nodeName:}" failed. No retries permitted until 2026-07-23 04:40:09.387916876 +0000 UTC m=+44.730429742 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/244b6204-6371-4924-815e-fd84477ce28d-metrics-certs") pod "router-default-657f4d9d48-7l4rd" (UID: "244b6204-6371-4924-815e-fd84477ce28d") : secret "router-metrics-certs-default" not found Jul 23 04:40:05.415823 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:05.415790 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4p4x8" event={"ID":"d8777622-488c-4307-a8fc-5d4af6137595","Type":"ContainerStarted","Data":"92c629a5f21cc0b0a84cc9143eca14b6fc216b0acf417468ff1dfd1b0ac8a9d3"} Jul 23 04:40:05.417312 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:05.417289 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-8v8xp" event={"ID":"aa0558ac-8329-4a03-8a88-76a2a42d49a1","Type":"ContainerStarted","Data":"1f4b3c4ac8f230a6509a51ba03c0d33825543db228471602cecafa5af61bff93"} Jul 23 04:40:05.433168 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:05.433131 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4p4x8" podStartSLOduration=1.960356513 podStartE2EDuration="4.433118472s" totalImagesPullingTime="2.472761959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:40:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:40:05.432810383 +0000 UTC m=+40.775323271" watchObservedRunningTime="2026-07-23 04:40:05.433118472 +0000 UTC m=+40.775631359" Jul 23 04:40:05.452687 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:05.452647 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-8v8xp" podStartSLOduration=1.887935142 podStartE2EDuration="4.452634685s" totalImagesPullingTime="2.564699543s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:40:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:40:05.451957832 +0000 UTC m=+40.794470720" watchObservedRunningTime="2026-07-23 04:40:05.452634685 +0000 UTC m=+40.795147590" Jul 23 04:40:06.556658 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:06.556622 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6k5dg"] Jul 23 04:40:06.574218 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:06.574187 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6k5dg" Jul 23 04:40:06.575422 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:06.575398 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6k5dg"] Jul 23 04:40:06.577339 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:06.577320 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4ss2x\"" Jul 23 04:40:06.577592 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:06.577571 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 23 04:40:06.578138 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:06.578123 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 23 04:40:06.699565 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:06.699526 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-data-volume\") pod \"insights-runtime-extractor-6k5dg\" (UID: \"b2a0526d-c1cd-47da-95e8-aaee5f6f094d\") " pod="openshift-insights/insights-runtime-extractor-6k5dg" Jul 23 04:40:06.699565 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:06.699565 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6k5dg\" (UID: \"b2a0526d-c1cd-47da-95e8-aaee5f6f094d\") " pod="openshift-insights/insights-runtime-extractor-6k5dg" Jul 23 04:40:06.699752 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:06.699586 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6k5dg\" (UID: \"b2a0526d-c1cd-47da-95e8-aaee5f6f094d\") " pod="openshift-insights/insights-runtime-extractor-6k5dg" Jul 23 04:40:06.699752 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:06.699680 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j67sl\" (UniqueName: \"kubernetes.io/projected/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-kube-api-access-j67sl\") pod \"insights-runtime-extractor-6k5dg\" (UID: \"b2a0526d-c1cd-47da-95e8-aaee5f6f094d\") " pod="openshift-insights/insights-runtime-extractor-6k5dg" Jul 23 04:40:06.699822 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:06.699762 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-crio-socket\") pod \"insights-runtime-extractor-6k5dg\" (UID: \"b2a0526d-c1cd-47da-95e8-aaee5f6f094d\") " pod="openshift-insights/insights-runtime-extractor-6k5dg" Jul 23 04:40:06.801035 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:06.801001 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-crio-socket\") pod \"insights-runtime-extractor-6k5dg\" (UID: \"b2a0526d-c1cd-47da-95e8-aaee5f6f094d\") " pod="openshift-insights/insights-runtime-extractor-6k5dg" Jul 23 04:40:06.801195 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:06.801065 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-data-volume\") pod \"insights-runtime-extractor-6k5dg\" (UID: \"b2a0526d-c1cd-47da-95e8-aaee5f6f094d\") " pod="openshift-insights/insights-runtime-extractor-6k5dg" Jul 23 04:40:06.801195 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:06.801085 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6k5dg\" (UID: \"b2a0526d-c1cd-47da-95e8-aaee5f6f094d\") " pod="openshift-insights/insights-runtime-extractor-6k5dg" Jul 23 04:40:06.801195 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:06.801108 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6k5dg\" (UID: \"b2a0526d-c1cd-47da-95e8-aaee5f6f094d\") " pod="openshift-insights/insights-runtime-extractor-6k5dg" Jul 23 04:40:06.801195 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:06.801169 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 04:40:06.801384 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:06.801204 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 04:40:06.801384 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:06.801226 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-kube-rbac-proxy-cm podName:b2a0526d-c1cd-47da-95e8-aaee5f6f094d nodeName:}" failed. No retries permitted until 2026-07-23 04:40:07.301212468 +0000 UTC m=+42.643725334 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6k5dg" (UID: "b2a0526d-c1cd-47da-95e8-aaee5f6f094d") : configmap "kube-rbac-proxy" not found Jul 23 04:40:06.801384 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:06.801262 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-insights-runtime-extractor-tls podName:b2a0526d-c1cd-47da-95e8-aaee5f6f094d nodeName:}" failed. No retries permitted until 2026-07-23 04:40:07.301243019 +0000 UTC m=+42.643755885 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6k5dg" (UID: "b2a0526d-c1cd-47da-95e8-aaee5f6f094d") : secret "insights-runtime-extractor-tls" not found Jul 23 04:40:06.801384 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:06.801265 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-crio-socket\") pod \"insights-runtime-extractor-6k5dg\" (UID: \"b2a0526d-c1cd-47da-95e8-aaee5f6f094d\") " pod="openshift-insights/insights-runtime-extractor-6k5dg" Jul 23 04:40:06.801384 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:06.801313 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j67sl\" (UniqueName: \"kubernetes.io/projected/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-kube-api-access-j67sl\") pod \"insights-runtime-extractor-6k5dg\" (UID: \"b2a0526d-c1cd-47da-95e8-aaee5f6f094d\") " pod="openshift-insights/insights-runtime-extractor-6k5dg" Jul 23 04:40:06.801384 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:06.801360 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-data-volume\") pod \"insights-runtime-extractor-6k5dg\" (UID: \"b2a0526d-c1cd-47da-95e8-aaee5f6f094d\") " pod="openshift-insights/insights-runtime-extractor-6k5dg" Jul 23 04:40:06.811315 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:06.811254 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j67sl\" (UniqueName: \"kubernetes.io/projected/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-kube-api-access-j67sl\") pod \"insights-runtime-extractor-6k5dg\" (UID: \"b2a0526d-c1cd-47da-95e8-aaee5f6f094d\") " pod="openshift-insights/insights-runtime-extractor-6k5dg" Jul 23 04:40:07.304200 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:07.304169 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6k5dg\" (UID: \"b2a0526d-c1cd-47da-95e8-aaee5f6f094d\") " pod="openshift-insights/insights-runtime-extractor-6k5dg" Jul 23 04:40:07.304200 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:07.304210 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6k5dg\" (UID: \"b2a0526d-c1cd-47da-95e8-aaee5f6f094d\") " pod="openshift-insights/insights-runtime-extractor-6k5dg" Jul 23 04:40:07.304506 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:07.304267 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/10491e7a-c5ab-4d40-a5ff-30d5714ffdf9-metrics-tls\") pod \"dns-default-45c4t\" (UID: \"10491e7a-c5ab-4d40-a5ff-30d5714ffdf9\") " pod="openshift-dns/dns-default-45c4t" Jul 23 04:40:07.304506 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:07.304338 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 04:40:07.304506 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:07.304361 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 04:40:07.304506 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:07.304367 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 04:40:07.304506 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:07.304420 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10491e7a-c5ab-4d40-a5ff-30d5714ffdf9-metrics-tls podName:10491e7a-c5ab-4d40-a5ff-30d5714ffdf9 nodeName:}" failed. No retries permitted until 2026-07-23 04:40:15.304401102 +0000 UTC m=+50.646913970 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/10491e7a-c5ab-4d40-a5ff-30d5714ffdf9-metrics-tls") pod "dns-default-45c4t" (UID: "10491e7a-c5ab-4d40-a5ff-30d5714ffdf9") : secret "dns-default-metrics-tls" not found Jul 23 04:40:07.304506 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:07.304434 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-kube-rbac-proxy-cm podName:b2a0526d-c1cd-47da-95e8-aaee5f6f094d nodeName:}" failed. No retries permitted until 2026-07-23 04:40:08.304428016 +0000 UTC m=+43.646940882 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6k5dg" (UID: "b2a0526d-c1cd-47da-95e8-aaee5f6f094d") : configmap "kube-rbac-proxy" not found Jul 23 04:40:07.304506 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:07.304445 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-insights-runtime-extractor-tls podName:b2a0526d-c1cd-47da-95e8-aaee5f6f094d nodeName:}" failed. No retries permitted until 2026-07-23 04:40:08.304440283 +0000 UTC m=+43.646953148 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6k5dg" (UID: "b2a0526d-c1cd-47da-95e8-aaee5f6f094d") : secret "insights-runtime-extractor-tls" not found Jul 23 04:40:08.313073 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:08.313039 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6k5dg\" (UID: \"b2a0526d-c1cd-47da-95e8-aaee5f6f094d\") " pod="openshift-insights/insights-runtime-extractor-6k5dg" Jul 23 04:40:08.313073 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:08.313079 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6k5dg\" (UID: \"b2a0526d-c1cd-47da-95e8-aaee5f6f094d\") " pod="openshift-insights/insights-runtime-extractor-6k5dg" Jul 23 04:40:08.313493 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:08.313172 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 04:40:08.313493 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:08.313207 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 04:40:08.313493 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:08.313242 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-kube-rbac-proxy-cm podName:b2a0526d-c1cd-47da-95e8-aaee5f6f094d nodeName:}" failed. No retries permitted until 2026-07-23 04:40:10.313222389 +0000 UTC m=+45.655735260 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6k5dg" (UID: "b2a0526d-c1cd-47da-95e8-aaee5f6f094d") : configmap "kube-rbac-proxy" not found Jul 23 04:40:08.313493 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:08.313263 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-insights-runtime-extractor-tls podName:b2a0526d-c1cd-47da-95e8-aaee5f6f094d nodeName:}" failed. No retries permitted until 2026-07-23 04:40:10.313251123 +0000 UTC m=+45.655764011 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6k5dg" (UID: "b2a0526d-c1cd-47da-95e8-aaee5f6f094d") : secret "insights-runtime-extractor-tls" not found Jul 23 04:40:08.551188 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:08.551164 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-57bmk_b9a49f0e-338a-47ac-90f7-281dae5538e4/dns-node-resolver/0.log" Jul 23 04:40:09.320911 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:09.320881 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d15db39a-c63b-4806-86a1-ba472f599e54-registry-tls\") pod \"image-registry-6798b7cd97-rbnww\" (UID: \"d15db39a-c63b-4806-86a1-ba472f599e54\") " pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" Jul 23 04:40:09.321268 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:09.321032 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 04:40:09.321268 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:09.321051 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6798b7cd97-rbnww: secret "image-registry-tls" not found Jul 23 04:40:09.321268 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:09.321104 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d15db39a-c63b-4806-86a1-ba472f599e54-registry-tls podName:d15db39a-c63b-4806-86a1-ba472f599e54 nodeName:}" failed. No retries permitted until 2026-07-23 04:40:17.321087886 +0000 UTC m=+52.663600783 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d15db39a-c63b-4806-86a1-ba472f599e54-registry-tls") pod "image-registry-6798b7cd97-rbnww" (UID: "d15db39a-c63b-4806-86a1-ba472f599e54") : secret "image-registry-tls" not found Jul 23 04:40:09.346544 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:09.346523 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-grlbc_d69de93b-d93f-4f10-b618-0cdcb545197a/node-ca/0.log" Jul 23 04:40:09.421883 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:09.421857 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/244b6204-6371-4924-815e-fd84477ce28d-service-ca-bundle\") pod \"router-default-657f4d9d48-7l4rd\" (UID: \"244b6204-6371-4924-815e-fd84477ce28d\") " pod="openshift-ingress/router-default-657f4d9d48-7l4rd" Jul 23 04:40:09.421997 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:09.421925 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/244b6204-6371-4924-815e-fd84477ce28d-metrics-certs\") pod \"router-default-657f4d9d48-7l4rd\" (UID: \"244b6204-6371-4924-815e-fd84477ce28d\") " pod="openshift-ingress/router-default-657f4d9d48-7l4rd" Jul 23 04:40:09.422041 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:09.422014 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/244b6204-6371-4924-815e-fd84477ce28d-service-ca-bundle podName:244b6204-6371-4924-815e-fd84477ce28d nodeName:}" failed. No retries permitted until 2026-07-23 04:40:17.421998017 +0000 UTC m=+52.764510883 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/244b6204-6371-4924-815e-fd84477ce28d-service-ca-bundle") pod "router-default-657f4d9d48-7l4rd" (UID: "244b6204-6371-4924-815e-fd84477ce28d") : configmap references non-existent config key: service-ca.crt Jul 23 04:40:09.422041 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:09.422016 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 04:40:09.422125 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:09.422063 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/244b6204-6371-4924-815e-fd84477ce28d-metrics-certs podName:244b6204-6371-4924-815e-fd84477ce28d nodeName:}" failed. No retries permitted until 2026-07-23 04:40:17.422051692 +0000 UTC m=+52.764564559 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/244b6204-6371-4924-815e-fd84477ce28d-metrics-certs") pod "router-default-657f4d9d48-7l4rd" (UID: "244b6204-6371-4924-815e-fd84477ce28d") : secret "router-metrics-certs-default" not found Jul 23 04:40:10.328537 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:10.328502 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6k5dg\" (UID: \"b2a0526d-c1cd-47da-95e8-aaee5f6f094d\") " pod="openshift-insights/insights-runtime-extractor-6k5dg" Jul 23 04:40:10.328905 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:10.328554 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6k5dg\" (UID: \"b2a0526d-c1cd-47da-95e8-aaee5f6f094d\") " pod="openshift-insights/insights-runtime-extractor-6k5dg" Jul 23 04:40:10.328905 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:10.328651 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 04:40:10.328905 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:10.328666 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 04:40:10.328905 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:10.328711 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-kube-rbac-proxy-cm podName:b2a0526d-c1cd-47da-95e8-aaee5f6f094d nodeName:}" failed. No retries permitted until 2026-07-23 04:40:14.328698239 +0000 UTC m=+49.671211105 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6k5dg" (UID: "b2a0526d-c1cd-47da-95e8-aaee5f6f094d") : configmap "kube-rbac-proxy" not found Jul 23 04:40:10.328905 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:10.328721 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-insights-runtime-extractor-tls podName:b2a0526d-c1cd-47da-95e8-aaee5f6f094d nodeName:}" failed. No retries permitted until 2026-07-23 04:40:14.328716102 +0000 UTC m=+49.671228967 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6k5dg" (UID: "b2a0526d-c1cd-47da-95e8-aaee5f6f094d") : secret "insights-runtime-extractor-tls" not found Jul 23 04:40:10.547188 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:10.547161 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qj5zw_8fb208f1-6626-4be0-9bfa-7cb8766d58ec/serve-healthcheck-canary/0.log" Jul 23 04:40:14.360892 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:14.360850 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6k5dg\" (UID: \"b2a0526d-c1cd-47da-95e8-aaee5f6f094d\") " pod="openshift-insights/insights-runtime-extractor-6k5dg" Jul 23 04:40:14.360892 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:14.360895 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6k5dg\" (UID: \"b2a0526d-c1cd-47da-95e8-aaee5f6f094d\") " pod="openshift-insights/insights-runtime-extractor-6k5dg" Jul 23 04:40:14.361348 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:14.360977 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 04:40:14.361348 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:14.361011 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 04:40:14.361348 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:14.361033 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-kube-rbac-proxy-cm podName:b2a0526d-c1cd-47da-95e8-aaee5f6f094d nodeName:}" failed. No retries permitted until 2026-07-23 04:40:22.361017295 +0000 UTC m=+57.703530161 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6k5dg" (UID: "b2a0526d-c1cd-47da-95e8-aaee5f6f094d") : configmap "kube-rbac-proxy" not found Jul 23 04:40:14.361348 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:14.361082 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-insights-runtime-extractor-tls podName:b2a0526d-c1cd-47da-95e8-aaee5f6f094d nodeName:}" failed. No retries permitted until 2026-07-23 04:40:22.361063385 +0000 UTC m=+57.703576268 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6k5dg" (UID: "b2a0526d-c1cd-47da-95e8-aaee5f6f094d") : secret "insights-runtime-extractor-tls" not found Jul 23 04:40:15.367569 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:15.367536 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/10491e7a-c5ab-4d40-a5ff-30d5714ffdf9-metrics-tls\") pod \"dns-default-45c4t\" (UID: \"10491e7a-c5ab-4d40-a5ff-30d5714ffdf9\") " pod="openshift-dns/dns-default-45c4t" Jul 23 04:40:15.369892 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:15.369876 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/10491e7a-c5ab-4d40-a5ff-30d5714ffdf9-metrics-tls\") pod \"dns-default-45c4t\" (UID: \"10491e7a-c5ab-4d40-a5ff-30d5714ffdf9\") " pod="openshift-dns/dns-default-45c4t" Jul 23 04:40:15.433320 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:15.433292 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-45c4t" Jul 23 04:40:15.591415 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:15.591386 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-45c4t"] Jul 23 04:40:15.595042 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:40:15.595015 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10491e7a_c5ab_4d40_a5ff_30d5714ffdf9.slice/crio-592fc455d422ac5ccadeb77582cfc1e56951256023188c48debb433b323f550f WatchSource:0}: Error finding container 592fc455d422ac5ccadeb77582cfc1e56951256023188c48debb433b323f550f: Status 404 returned error can't find the container with id 592fc455d422ac5ccadeb77582cfc1e56951256023188c48debb433b323f550f Jul 23 04:40:16.448417 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:16.448377 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-45c4t" event={"ID":"10491e7a-c5ab-4d40-a5ff-30d5714ffdf9","Type":"ContainerStarted","Data":"592fc455d422ac5ccadeb77582cfc1e56951256023188c48debb433b323f550f"} Jul 23 04:40:17.382062 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:17.382020 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d15db39a-c63b-4806-86a1-ba472f599e54-registry-tls\") pod \"image-registry-6798b7cd97-rbnww\" (UID: \"d15db39a-c63b-4806-86a1-ba472f599e54\") " pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" Jul 23 04:40:17.384935 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:17.384903 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d15db39a-c63b-4806-86a1-ba472f599e54-registry-tls\") pod \"image-registry-6798b7cd97-rbnww\" (UID: \"d15db39a-c63b-4806-86a1-ba472f599e54\") " pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" Jul 23 04:40:17.433122 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:17.433083 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" Jul 23 04:40:17.482545 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:17.482518 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/244b6204-6371-4924-815e-fd84477ce28d-metrics-certs\") pod \"router-default-657f4d9d48-7l4rd\" (UID: \"244b6204-6371-4924-815e-fd84477ce28d\") " pod="openshift-ingress/router-default-657f4d9d48-7l4rd" Jul 23 04:40:17.482969 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:17.482609 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/244b6204-6371-4924-815e-fd84477ce28d-service-ca-bundle\") pod \"router-default-657f4d9d48-7l4rd\" (UID: \"244b6204-6371-4924-815e-fd84477ce28d\") " pod="openshift-ingress/router-default-657f4d9d48-7l4rd" Jul 23 04:40:17.482969 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:17.482813 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/244b6204-6371-4924-815e-fd84477ce28d-service-ca-bundle podName:244b6204-6371-4924-815e-fd84477ce28d nodeName:}" failed. No retries permitted until 2026-07-23 04:40:33.482787465 +0000 UTC m=+68.825300351 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/244b6204-6371-4924-815e-fd84477ce28d-service-ca-bundle") pod "router-default-657f4d9d48-7l4rd" (UID: "244b6204-6371-4924-815e-fd84477ce28d") : configmap references non-existent config key: service-ca.crt Jul 23 04:40:17.485216 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:17.485192 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/244b6204-6371-4924-815e-fd84477ce28d-metrics-certs\") pod \"router-default-657f4d9d48-7l4rd\" (UID: \"244b6204-6371-4924-815e-fd84477ce28d\") " pod="openshift-ingress/router-default-657f4d9d48-7l4rd" Jul 23 04:40:17.590120 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:17.590099 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6798b7cd97-rbnww"] Jul 23 04:40:17.595025 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:40:17.594998 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd15db39a_c63b_4806_86a1_ba472f599e54.slice/crio-deecfe25cf86443665ae0367f36132131593bc9b648039cf6f98975da4822ea0 WatchSource:0}: Error finding container deecfe25cf86443665ae0367f36132131593bc9b648039cf6f98975da4822ea0: Status 404 returned error can't find the container with id deecfe25cf86443665ae0367f36132131593bc9b648039cf6f98975da4822ea0 Jul 23 04:40:18.459021 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:18.458981 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-45c4t" event={"ID":"10491e7a-c5ab-4d40-a5ff-30d5714ffdf9","Type":"ContainerStarted","Data":"2706d54b3e2aa0ff4c90e17be674bbce5e4ceaf0397d18cffc0ac4fa4c8fb994"} Jul 23 04:40:18.459183 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:18.459051 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-45c4t" event={"ID":"10491e7a-c5ab-4d40-a5ff-30d5714ffdf9","Type":"ContainerStarted","Data":"8e2aa3b15eb0958984defd045140c68a9a115894ca5a4b68f6a485390fe583de"} Jul 23 04:40:18.459183 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:18.459080 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-45c4t" Jul 23 04:40:18.460390 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:18.460364 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" event={"ID":"d15db39a-c63b-4806-86a1-ba472f599e54","Type":"ContainerStarted","Data":"33f7fdcbb91ec750405439c4a033aadcefd6f3b5087e109aad11ca28c69b88ec"} Jul 23 04:40:18.460497 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:18.460407 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" event={"ID":"d15db39a-c63b-4806-86a1-ba472f599e54","Type":"ContainerStarted","Data":"deecfe25cf86443665ae0367f36132131593bc9b648039cf6f98975da4822ea0"} Jul 23 04:40:18.460497 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:18.460477 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" Jul 23 04:40:18.481055 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:18.479006 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-45c4t" podStartSLOduration=17.584888402 podStartE2EDuration="19.478988438s" totalImagesPullingTime="1.894100036s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:39:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:40:18.477309785 +0000 UTC m=+53.819822672" watchObservedRunningTime="2026-07-23 04:40:18.478988438 +0000 UTC m=+53.821501343" Jul 23 04:40:18.498027 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:18.497975 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" podStartSLOduration=17.497964353 podStartE2EDuration="17.497964353s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:40:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:40:18.49747909 +0000 UTC m=+53.839991969" watchObservedRunningTime="2026-07-23 04:40:18.497964353 +0000 UTC m=+53.840477240" Jul 23 04:40:22.421371 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:22.421337 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6k5dg\" (UID: \"b2a0526d-c1cd-47da-95e8-aaee5f6f094d\") " pod="openshift-insights/insights-runtime-extractor-6k5dg" Jul 23 04:40:22.421371 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:22.421374 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6k5dg\" (UID: \"b2a0526d-c1cd-47da-95e8-aaee5f6f094d\") " pod="openshift-insights/insights-runtime-extractor-6k5dg" Jul 23 04:40:22.422019 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:22.421990 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6k5dg\" (UID: \"b2a0526d-c1cd-47da-95e8-aaee5f6f094d\") " pod="openshift-insights/insights-runtime-extractor-6k5dg" Jul 23 04:40:22.423583 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:22.423564 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b2a0526d-c1cd-47da-95e8-aaee5f6f094d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6k5dg\" (UID: \"b2a0526d-c1cd-47da-95e8-aaee5f6f094d\") " pod="openshift-insights/insights-runtime-extractor-6k5dg" Jul 23 04:40:22.483688 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:22.483660 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6k5dg" Jul 23 04:40:22.615884 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:22.615858 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6k5dg"] Jul 23 04:40:22.619495 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:40:22.619466 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2a0526d_c1cd_47da_95e8_aaee5f6f094d.slice/crio-ac1d2274c8ed74f31cc8c91cee3d15a17c63b1c7b4c696a3c0f4631eb1b6dbad WatchSource:0}: Error finding container ac1d2274c8ed74f31cc8c91cee3d15a17c63b1c7b4c696a3c0f4631eb1b6dbad: Status 404 returned error can't find the container with id ac1d2274c8ed74f31cc8c91cee3d15a17c63b1c7b4c696a3c0f4631eb1b6dbad Jul 23 04:40:23.478652 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:23.478607 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6k5dg" event={"ID":"b2a0526d-c1cd-47da-95e8-aaee5f6f094d","Type":"ContainerStarted","Data":"ac1d2274c8ed74f31cc8c91cee3d15a17c63b1c7b4c696a3c0f4631eb1b6dbad"} Jul 23 04:40:25.382489 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:25.382452 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vphbq" Jul 23 04:40:28.465903 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:28.465870 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-45c4t" Jul 23 04:40:28.497330 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:28.497291 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6k5dg" event={"ID":"b2a0526d-c1cd-47da-95e8-aaee5f6f094d","Type":"ContainerStarted","Data":"2d0b536f54a95a37d10016f63eaac70b60ffed98206b3ea9d0dd1906dc677347"} Jul 23 04:40:31.000661 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:31.000616 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dab60874-8b3b-411d-b58e-4ec857bc9629-metrics-certs\") pod \"network-metrics-daemon-mpnqv\" (UID: \"dab60874-8b3b-411d-b58e-4ec857bc9629\") " pod="openshift-multus/network-metrics-daemon-mpnqv" Jul 23 04:40:31.001178 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:31.000730 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/555665cd-6364-4d9f-8326-4973b140cc64-original-pull-secret\") pod \"global-pull-secret-syncer-cbzpv\" (UID: \"555665cd-6364-4d9f-8326-4973b140cc64\") " pod="kube-system/global-pull-secret-syncer-cbzpv" Jul 23 04:40:31.003580 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:31.003526 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 23 04:40:31.003694 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:31.003596 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 23 04:40:31.013949 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:31.013923 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/555665cd-6364-4d9f-8326-4973b140cc64-original-pull-secret\") pod \"global-pull-secret-syncer-cbzpv\" (UID: \"555665cd-6364-4d9f-8326-4973b140cc64\") " pod="kube-system/global-pull-secret-syncer-cbzpv" Jul 23 04:40:31.014070 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:31.014051 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dab60874-8b3b-411d-b58e-4ec857bc9629-metrics-certs\") pod \"network-metrics-daemon-mpnqv\" (UID: \"dab60874-8b3b-411d-b58e-4ec857bc9629\") " pod="openshift-multus/network-metrics-daemon-mpnqv" Jul 23 04:40:31.101276 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:31.101241 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czjdx\" (UniqueName: \"kubernetes.io/projected/6346ee84-530c-41f1-9f8a-6d83f79daa5e-kube-api-access-czjdx\") pod \"network-check-target-7dsv7\" (UID: \"6346ee84-530c-41f1-9f8a-6d83f79daa5e\") " pod="openshift-network-diagnostics/network-check-target-7dsv7" Jul 23 04:40:31.104015 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:31.103996 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 23 04:40:31.114280 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:31.114258 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 23 04:40:31.124996 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:31.124971 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czjdx\" (UniqueName: \"kubernetes.io/projected/6346ee84-530c-41f1-9f8a-6d83f79daa5e-kube-api-access-czjdx\") pod \"network-check-target-7dsv7\" (UID: \"6346ee84-530c-41f1-9f8a-6d83f79daa5e\") " pod="openshift-network-diagnostics/network-check-target-7dsv7" Jul 23 04:40:31.214696 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:31.214676 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbzpv" Jul 23 04:40:31.228019 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:31.223854 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-m89hb\"" Jul 23 04:40:31.230590 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:31.230571 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j2njx\"" Jul 23 04:40:31.231589 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:31.231568 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7dsv7" Jul 23 04:40:31.238426 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:31.238272 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mpnqv" Jul 23 04:40:31.365955 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:31.365781 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cbzpv"] Jul 23 04:40:31.369686 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:40:31.369652 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod555665cd_6364_4d9f_8326_4973b140cc64.slice/crio-7a55fcf5cd45ce30b49283b8001b307f7125c2c0a5ebed2c6138c77717fc1637 WatchSource:0}: Error finding container 7a55fcf5cd45ce30b49283b8001b307f7125c2c0a5ebed2c6138c77717fc1637: Status 404 returned error can't find the container with id 7a55fcf5cd45ce30b49283b8001b307f7125c2c0a5ebed2c6138c77717fc1637 Jul 23 04:40:31.383247 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:31.383220 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7dsv7"] Jul 23 04:40:31.385704 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:40:31.385679 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6346ee84_530c_41f1_9f8a_6d83f79daa5e.slice/crio-6fc9f79594ea390c4023150479e8663eb956dd4c02a19a46fb3797d4c8362b08 WatchSource:0}: Error finding container 6fc9f79594ea390c4023150479e8663eb956dd4c02a19a46fb3797d4c8362b08: Status 404 returned error can't find the container with id 6fc9f79594ea390c4023150479e8663eb956dd4c02a19a46fb3797d4c8362b08 Jul 23 04:40:31.403165 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:31.403136 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mpnqv"] Jul 23 04:40:31.406546 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:40:31.406524 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddab60874_8b3b_411d_b58e_4ec857bc9629.slice/crio-ae70cc9eae54461881211945f1cecad97cee761c16b7c159ad86341b321fdb30 WatchSource:0}: Error finding container ae70cc9eae54461881211945f1cecad97cee761c16b7c159ad86341b321fdb30: Status 404 returned error can't find the container with id ae70cc9eae54461881211945f1cecad97cee761c16b7c159ad86341b321fdb30 Jul 23 04:40:31.508208 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:31.508124 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6k5dg" event={"ID":"b2a0526d-c1cd-47da-95e8-aaee5f6f094d","Type":"ContainerStarted","Data":"0ed0ca9649fcbbf807de5ef738acb0ce63fa3a45fb26ff480c39c1bf686a3ae2"} Jul 23 04:40:31.509310 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:31.509284 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7dsv7" event={"ID":"6346ee84-530c-41f1-9f8a-6d83f79daa5e","Type":"ContainerStarted","Data":"6fc9f79594ea390c4023150479e8663eb956dd4c02a19a46fb3797d4c8362b08"} Jul 23 04:40:31.510448 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:31.510427 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mpnqv" event={"ID":"dab60874-8b3b-411d-b58e-4ec857bc9629","Type":"ContainerStarted","Data":"ae70cc9eae54461881211945f1cecad97cee761c16b7c159ad86341b321fdb30"} Jul 23 04:40:31.511520 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:31.511499 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cbzpv" event={"ID":"555665cd-6364-4d9f-8326-4973b140cc64","Type":"ContainerStarted","Data":"7a55fcf5cd45ce30b49283b8001b307f7125c2c0a5ebed2c6138c77717fc1637"} Jul 23 04:40:32.659066 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:32.658906 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6798b7cd97-rbnww"] Jul 23 04:40:32.667015 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:32.666808 2562 patch_prober.go:28] interesting pod/image-registry-6798b7cd97-rbnww container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 23 04:40:32.667015 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:32.666898 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" podUID="d15db39a-c63b-4806-86a1-ba472f599e54" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 04:40:32.711487 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:32.710813 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-pzvsl"] Jul 23 04:40:32.714817 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:32.714046 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-pzvsl" Jul 23 04:40:32.718650 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:32.717468 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 23 04:40:32.718650 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:32.718193 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-tvh8n\"" Jul 23 04:40:32.718650 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:32.718484 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 23 04:40:32.730518 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:32.730494 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-pzvsl"] Jul 23 04:40:32.815228 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:32.814110 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5f446fbc9d-hzvct"] Jul 23 04:40:32.816933 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:32.816851 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9d5z\" (UniqueName: \"kubernetes.io/projected/1a3d4f1d-65d6-4abd-a649-6353217d4c77-kube-api-access-c9d5z\") pod \"downloads-5bc67dcd46-pzvsl\" (UID: \"1a3d4f1d-65d6-4abd-a649-6353217d4c77\") " pod="openshift-console/downloads-5bc67dcd46-pzvsl" Jul 23 04:40:32.817802 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:32.817253 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f446fbc9d-hzvct" Jul 23 04:40:32.837499 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:32.837427 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5f446fbc9d-hzvct"] Jul 23 04:40:32.918300 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:32.917889 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9d5z\" (UniqueName: \"kubernetes.io/projected/1a3d4f1d-65d6-4abd-a649-6353217d4c77-kube-api-access-c9d5z\") pod \"downloads-5bc67dcd46-pzvsl\" (UID: \"1a3d4f1d-65d6-4abd-a649-6353217d4c77\") " pod="openshift-console/downloads-5bc67dcd46-pzvsl" Jul 23 04:40:32.918300 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:32.917941 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d21b024b-02b0-4c6a-8d5c-36a20694fea7-bound-sa-token\") pod \"image-registry-5f446fbc9d-hzvct\" (UID: \"d21b024b-02b0-4c6a-8d5c-36a20694fea7\") " pod="openshift-image-registry/image-registry-5f446fbc9d-hzvct" Jul 23 04:40:32.918300 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:32.917980 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d21b024b-02b0-4c6a-8d5c-36a20694fea7-installation-pull-secrets\") pod \"image-registry-5f446fbc9d-hzvct\" (UID: \"d21b024b-02b0-4c6a-8d5c-36a20694fea7\") " pod="openshift-image-registry/image-registry-5f446fbc9d-hzvct" Jul 23 04:40:32.918300 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:32.918047 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d21b024b-02b0-4c6a-8d5c-36a20694fea7-ca-trust-extracted\") pod \"image-registry-5f446fbc9d-hzvct\" (UID: \"d21b024b-02b0-4c6a-8d5c-36a20694fea7\") " pod="openshift-image-registry/image-registry-5f446fbc9d-hzvct" Jul 23 04:40:32.918300 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:32.918081 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d21b024b-02b0-4c6a-8d5c-36a20694fea7-trusted-ca\") pod \"image-registry-5f446fbc9d-hzvct\" (UID: \"d21b024b-02b0-4c6a-8d5c-36a20694fea7\") " pod="openshift-image-registry/image-registry-5f446fbc9d-hzvct" Jul 23 04:40:32.918300 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:32.918107 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d21b024b-02b0-4c6a-8d5c-36a20694fea7-registry-tls\") pod \"image-registry-5f446fbc9d-hzvct\" (UID: \"d21b024b-02b0-4c6a-8d5c-36a20694fea7\") " pod="openshift-image-registry/image-registry-5f446fbc9d-hzvct" Jul 23 04:40:32.918300 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:32.918136 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d21b024b-02b0-4c6a-8d5c-36a20694fea7-registry-certificates\") pod \"image-registry-5f446fbc9d-hzvct\" (UID: \"d21b024b-02b0-4c6a-8d5c-36a20694fea7\") " pod="openshift-image-registry/image-registry-5f446fbc9d-hzvct" Jul 23 04:40:32.918300 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:32.918160 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bdsl\" (UniqueName: \"kubernetes.io/projected/d21b024b-02b0-4c6a-8d5c-36a20694fea7-kube-api-access-9bdsl\") pod \"image-registry-5f446fbc9d-hzvct\" (UID: \"d21b024b-02b0-4c6a-8d5c-36a20694fea7\") " pod="openshift-image-registry/image-registry-5f446fbc9d-hzvct" Jul 23 04:40:32.918300 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:32.918209 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d21b024b-02b0-4c6a-8d5c-36a20694fea7-image-registry-private-configuration\") pod \"image-registry-5f446fbc9d-hzvct\" (UID: \"d21b024b-02b0-4c6a-8d5c-36a20694fea7\") " pod="openshift-image-registry/image-registry-5f446fbc9d-hzvct" Jul 23 04:40:32.935795 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:32.935736 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9d5z\" (UniqueName: \"kubernetes.io/projected/1a3d4f1d-65d6-4abd-a649-6353217d4c77-kube-api-access-c9d5z\") pod \"downloads-5bc67dcd46-pzvsl\" (UID: \"1a3d4f1d-65d6-4abd-a649-6353217d4c77\") " pod="openshift-console/downloads-5bc67dcd46-pzvsl" Jul 23 04:40:33.019161 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:33.019122 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d21b024b-02b0-4c6a-8d5c-36a20694fea7-ca-trust-extracted\") pod \"image-registry-5f446fbc9d-hzvct\" (UID: \"d21b024b-02b0-4c6a-8d5c-36a20694fea7\") " pod="openshift-image-registry/image-registry-5f446fbc9d-hzvct" Jul 23 04:40:33.019339 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:33.019177 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d21b024b-02b0-4c6a-8d5c-36a20694fea7-trusted-ca\") pod \"image-registry-5f446fbc9d-hzvct\" (UID: \"d21b024b-02b0-4c6a-8d5c-36a20694fea7\") " pod="openshift-image-registry/image-registry-5f446fbc9d-hzvct" Jul 23 04:40:33.019339 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:33.019205 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d21b024b-02b0-4c6a-8d5c-36a20694fea7-registry-tls\") pod \"image-registry-5f446fbc9d-hzvct\" (UID: \"d21b024b-02b0-4c6a-8d5c-36a20694fea7\") " pod="openshift-image-registry/image-registry-5f446fbc9d-hzvct" Jul 23 04:40:33.019339 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:33.019235 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d21b024b-02b0-4c6a-8d5c-36a20694fea7-registry-certificates\") pod \"image-registry-5f446fbc9d-hzvct\" (UID: \"d21b024b-02b0-4c6a-8d5c-36a20694fea7\") " pod="openshift-image-registry/image-registry-5f446fbc9d-hzvct" Jul 23 04:40:33.019339 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:33.019262 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bdsl\" (UniqueName: \"kubernetes.io/projected/d21b024b-02b0-4c6a-8d5c-36a20694fea7-kube-api-access-9bdsl\") pod \"image-registry-5f446fbc9d-hzvct\" (UID: \"d21b024b-02b0-4c6a-8d5c-36a20694fea7\") " pod="openshift-image-registry/image-registry-5f446fbc9d-hzvct" Jul 23 04:40:33.019339 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:33.019292 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d21b024b-02b0-4c6a-8d5c-36a20694fea7-image-registry-private-configuration\") pod \"image-registry-5f446fbc9d-hzvct\" (UID: \"d21b024b-02b0-4c6a-8d5c-36a20694fea7\") " pod="openshift-image-registry/image-registry-5f446fbc9d-hzvct" Jul 23 04:40:33.019339 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:33.019329 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d21b024b-02b0-4c6a-8d5c-36a20694fea7-bound-sa-token\") pod \"image-registry-5f446fbc9d-hzvct\" (UID: \"d21b024b-02b0-4c6a-8d5c-36a20694fea7\") " pod="openshift-image-registry/image-registry-5f446fbc9d-hzvct" Jul 23 04:40:33.019645 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:33.019365 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d21b024b-02b0-4c6a-8d5c-36a20694fea7-installation-pull-secrets\") pod \"image-registry-5f446fbc9d-hzvct\" (UID: \"d21b024b-02b0-4c6a-8d5c-36a20694fea7\") " pod="openshift-image-registry/image-registry-5f446fbc9d-hzvct" Jul 23 04:40:33.019645 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:33.019560 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d21b024b-02b0-4c6a-8d5c-36a20694fea7-ca-trust-extracted\") pod \"image-registry-5f446fbc9d-hzvct\" (UID: \"d21b024b-02b0-4c6a-8d5c-36a20694fea7\") " pod="openshift-image-registry/image-registry-5f446fbc9d-hzvct" Jul 23 04:40:33.020901 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:33.020857 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d21b024b-02b0-4c6a-8d5c-36a20694fea7-trusted-ca\") pod \"image-registry-5f446fbc9d-hzvct\" (UID: \"d21b024b-02b0-4c6a-8d5c-36a20694fea7\") " pod="openshift-image-registry/image-registry-5f446fbc9d-hzvct" Jul 23 04:40:33.021363 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:33.021335 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d21b024b-02b0-4c6a-8d5c-36a20694fea7-registry-certificates\") pod \"image-registry-5f446fbc9d-hzvct\" (UID: \"d21b024b-02b0-4c6a-8d5c-36a20694fea7\") " pod="openshift-image-registry/image-registry-5f446fbc9d-hzvct" Jul 23 04:40:33.025596 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:33.025179 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d21b024b-02b0-4c6a-8d5c-36a20694fea7-registry-tls\") pod \"image-registry-5f446fbc9d-hzvct\" (UID: \"d21b024b-02b0-4c6a-8d5c-36a20694fea7\") " pod="openshift-image-registry/image-registry-5f446fbc9d-hzvct" Jul 23 04:40:33.025596 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:33.025558 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d21b024b-02b0-4c6a-8d5c-36a20694fea7-image-registry-private-configuration\") pod \"image-registry-5f446fbc9d-hzvct\" (UID: \"d21b024b-02b0-4c6a-8d5c-36a20694fea7\") " pod="openshift-image-registry/image-registry-5f446fbc9d-hzvct" Jul 23 04:40:33.026996 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:33.026763 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-pzvsl" Jul 23 04:40:33.027646 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:33.027609 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d21b024b-02b0-4c6a-8d5c-36a20694fea7-installation-pull-secrets\") pod \"image-registry-5f446fbc9d-hzvct\" (UID: \"d21b024b-02b0-4c6a-8d5c-36a20694fea7\") " pod="openshift-image-registry/image-registry-5f446fbc9d-hzvct" Jul 23 04:40:33.047272 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:33.047244 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bdsl\" (UniqueName: \"kubernetes.io/projected/d21b024b-02b0-4c6a-8d5c-36a20694fea7-kube-api-access-9bdsl\") pod \"image-registry-5f446fbc9d-hzvct\" (UID: \"d21b024b-02b0-4c6a-8d5c-36a20694fea7\") " pod="openshift-image-registry/image-registry-5f446fbc9d-hzvct" Jul 23 04:40:33.050128 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:33.050088 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d21b024b-02b0-4c6a-8d5c-36a20694fea7-bound-sa-token\") pod \"image-registry-5f446fbc9d-hzvct\" (UID: \"d21b024b-02b0-4c6a-8d5c-36a20694fea7\") " pod="openshift-image-registry/image-registry-5f446fbc9d-hzvct" Jul 23 04:40:33.132287 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:33.132013 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f446fbc9d-hzvct" Jul 23 04:40:33.219848 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:33.219804 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-pzvsl"] Jul 23 04:40:33.220597 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:40:33.220567 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a3d4f1d_65d6_4abd_a649_6353217d4c77.slice/crio-55517fa3cc4269a33b660fb7e2c74eaaec1661a46ee5a5ae411623124f57269a WatchSource:0}: Error finding container 55517fa3cc4269a33b660fb7e2c74eaaec1661a46ee5a5ae411623124f57269a: Status 404 returned error can't find the container with id 55517fa3cc4269a33b660fb7e2c74eaaec1661a46ee5a5ae411623124f57269a Jul 23 04:40:33.310787 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:33.310749 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5f446fbc9d-hzvct"] Jul 23 04:40:33.316533 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:40:33.316502 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd21b024b_02b0_4c6a_8d5c_36a20694fea7.slice/crio-403d7825ca514714dd6fe2f955d7feb788560c90eb291ed9c28d97761391cf23 WatchSource:0}: Error finding container 403d7825ca514714dd6fe2f955d7feb788560c90eb291ed9c28d97761391cf23: Status 404 returned error can't find the container with id 403d7825ca514714dd6fe2f955d7feb788560c90eb291ed9c28d97761391cf23 Jul 23 04:40:33.522308 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:33.522224 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-pzvsl" event={"ID":"1a3d4f1d-65d6-4abd-a649-6353217d4c77","Type":"ContainerStarted","Data":"55517fa3cc4269a33b660fb7e2c74eaaec1661a46ee5a5ae411623124f57269a"} Jul 23 04:40:33.525117 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:33.524313 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/244b6204-6371-4924-815e-fd84477ce28d-service-ca-bundle\") pod \"router-default-657f4d9d48-7l4rd\" (UID: \"244b6204-6371-4924-815e-fd84477ce28d\") " pod="openshift-ingress/router-default-657f4d9d48-7l4rd" Jul 23 04:40:33.525117 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:33.525077 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/244b6204-6371-4924-815e-fd84477ce28d-service-ca-bundle\") pod \"router-default-657f4d9d48-7l4rd\" (UID: \"244b6204-6371-4924-815e-fd84477ce28d\") " pod="openshift-ingress/router-default-657f4d9d48-7l4rd" Jul 23 04:40:33.526725 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:33.526695 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f446fbc9d-hzvct" event={"ID":"d21b024b-02b0-4c6a-8d5c-36a20694fea7","Type":"ContainerStarted","Data":"30dbd416ff2c8f046f77d15fe2d40560512792b1e904dec362e5745ad70487ab"} Jul 23 04:40:33.526827 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:33.526746 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f446fbc9d-hzvct" event={"ID":"d21b024b-02b0-4c6a-8d5c-36a20694fea7","Type":"ContainerStarted","Data":"403d7825ca514714dd6fe2f955d7feb788560c90eb291ed9c28d97761391cf23"} Jul 23 04:40:33.527357 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:33.527338 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5f446fbc9d-hzvct" Jul 23 04:40:33.530724 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:33.530293 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mpnqv" event={"ID":"dab60874-8b3b-411d-b58e-4ec857bc9629","Type":"ContainerStarted","Data":"53272a31a9f19ab77e75b7e0ed3e19b8558b7229e8e718309181dca038c3d755"} Jul 23 04:40:33.530724 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:33.530341 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mpnqv" event={"ID":"dab60874-8b3b-411d-b58e-4ec857bc9629","Type":"ContainerStarted","Data":"d4fb57cb695890c481740b72982f2089a507d33f2ba20958f8b6e6ac63eb7c29"} Jul 23 04:40:33.550278 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:33.550226 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5f446fbc9d-hzvct" podStartSLOduration=1.550210688 podStartE2EDuration="1.550210688s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:40:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:40:33.549532478 +0000 UTC m=+68.892045365" watchObservedRunningTime="2026-07-23 04:40:33.550210688 +0000 UTC m=+68.892723576" Jul 23 04:40:33.568870 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:33.568539 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-mpnqv" podStartSLOduration=67.333841311 podStartE2EDuration="1m8.568520327s" totalImagesPullingTime="1.234679016s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:39:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:40:33.567217031 +0000 UTC m=+68.909729919" watchObservedRunningTime="2026-07-23 04:40:33.568520327 +0000 UTC m=+68.911033216" Jul 23 04:40:33.748096 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:33.748062 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-vfqhj\"" Jul 23 04:40:33.755660 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:33.755339 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-657f4d9d48-7l4rd" Jul 23 04:40:33.959039 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:33.958951 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-657f4d9d48-7l4rd"] Jul 23 04:40:35.996322 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:40:35.996274 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod244b6204_6371_4924_815e_fd84477ce28d.slice/crio-e9c8e515e632ad379455251b3c36e3d0af61354187a7ecf46e4f376689883066 WatchSource:0}: Error finding container e9c8e515e632ad379455251b3c36e3d0af61354187a7ecf46e4f376689883066: Status 404 returned error can't find the container with id e9c8e515e632ad379455251b3c36e3d0af61354187a7ecf46e4f376689883066 Jul 23 04:40:36.545656 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:36.545612 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-657f4d9d48-7l4rd" event={"ID":"244b6204-6371-4924-815e-fd84477ce28d","Type":"ContainerStarted","Data":"e9c8e515e632ad379455251b3c36e3d0af61354187a7ecf46e4f376689883066"} Jul 23 04:40:39.559525 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:39.559441 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7dsv7" event={"ID":"6346ee84-530c-41f1-9f8a-6d83f79daa5e","Type":"ContainerStarted","Data":"ce648a342e9753d05b211f73e6892f08174d23d25b1ebb0858686dc40715b586"} Jul 23 04:40:39.559945 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:39.559564 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7dsv7" Jul 23 04:40:39.561299 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:39.561258 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cbzpv" event={"ID":"555665cd-6364-4d9f-8326-4973b140cc64","Type":"ContainerStarted","Data":"94c7cd7531dfc96e8b4a166be6f47eb471d017b85cdac00f27aab03265604959"} Jul 23 04:40:39.563539 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:39.563517 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6k5dg" event={"ID":"b2a0526d-c1cd-47da-95e8-aaee5f6f094d","Type":"ContainerStarted","Data":"647704d564fcbf21235db9cc873fe8a86f758985edc7ddf188a9b9b8f308ecce"} Jul 23 04:40:39.564985 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:39.564963 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-657f4d9d48-7l4rd" event={"ID":"244b6204-6371-4924-815e-fd84477ce28d","Type":"ContainerStarted","Data":"413ffbfc864b8baac4e9ed01c2f9a260f0830943da5de4fc90fa1481d13d24b5"} Jul 23 04:40:39.582330 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:39.582275 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-7dsv7" podStartSLOduration=66.838766986 podStartE2EDuration="1m14.582261384s" totalImagesPullingTime="7.743494398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:39:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:40:39.580533252 +0000 UTC m=+74.923046142" watchObservedRunningTime="2026-07-23 04:40:39.582261384 +0000 UTC m=+74.924774272" Jul 23 04:40:39.597674 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:39.597625 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-cbzpv" podStartSLOduration=66.835592488 podStartE2EDuration="1m14.597607992s" totalImagesPullingTime="7.762015504s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 04:39:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:40:39.596414733 +0000 UTC m=+74.938927621" watchObservedRunningTime="2026-07-23 04:40:39.597607992 +0000 UTC m=+74.940120864" Jul 23 04:40:39.617753 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:39.617687 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-657f4d9d48-7l4rd" podStartSLOduration=38.617674607 podStartE2EDuration="38.617674607s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:40:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:40:39.616560403 +0000 UTC m=+74.959073295" watchObservedRunningTime="2026-07-23 04:40:39.617674607 +0000 UTC m=+74.960187495" Jul 23 04:40:39.636947 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:39.636519 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-6k5dg" podStartSLOduration=17.270964187 podStartE2EDuration="33.636499467s" totalImagesPullingTime="16.36553528s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 04:40:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:40:39.636212638 +0000 UTC m=+74.978725530" watchObservedRunningTime="2026-07-23 04:40:39.636499467 +0000 UTC m=+74.979012356" Jul 23 04:40:39.756625 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:39.756591 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-657f4d9d48-7l4rd" Jul 23 04:40:39.759795 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:39.759770 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-657f4d9d48-7l4rd" Jul 23 04:40:40.569357 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:40.569304 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-657f4d9d48-7l4rd" Jul 23 04:40:40.570635 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:40.570612 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-657f4d9d48-7l4rd" Jul 23 04:40:42.664670 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:42.664634 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" Jul 23 04:40:44.944693 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:44.944651 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-jpzmn"] Jul 23 04:40:44.948662 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:44.948629 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-jpzmn" Jul 23 04:40:44.952428 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:44.952202 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 23 04:40:44.952428 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:44.952204 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 23 04:40:44.952428 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:44.952252 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 23 04:40:44.952428 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:44.952252 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 23 04:40:44.952428 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:44.952277 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 23 04:40:44.952428 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:44.952311 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-l9zk9\"" Jul 23 04:40:44.958404 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:44.958378 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-jpzmn"] Jul 23 04:40:45.133868 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.133707 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3d7f6ab8-9a28-4631-a93c-726bf5ed7e18-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-jpzmn\" (UID: \"3d7f6ab8-9a28-4631-a93c-726bf5ed7e18\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jpzmn" Jul 23 04:40:45.133868 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.133764 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3d7f6ab8-9a28-4631-a93c-726bf5ed7e18-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-jpzmn\" (UID: \"3d7f6ab8-9a28-4631-a93c-726bf5ed7e18\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jpzmn" Jul 23 04:40:45.133868 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.133800 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3d7f6ab8-9a28-4631-a93c-726bf5ed7e18-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-jpzmn\" (UID: \"3d7f6ab8-9a28-4631-a93c-726bf5ed7e18\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jpzmn" Jul 23 04:40:45.134105 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.133908 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vqpr\" (UniqueName: \"kubernetes.io/projected/3d7f6ab8-9a28-4631-a93c-726bf5ed7e18-kube-api-access-4vqpr\") pod \"prometheus-operator-5945b5cfd-jpzmn\" (UID: \"3d7f6ab8-9a28-4631-a93c-726bf5ed7e18\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jpzmn" Jul 23 04:40:45.234600 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.234522 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vqpr\" (UniqueName: \"kubernetes.io/projected/3d7f6ab8-9a28-4631-a93c-726bf5ed7e18-kube-api-access-4vqpr\") pod \"prometheus-operator-5945b5cfd-jpzmn\" (UID: \"3d7f6ab8-9a28-4631-a93c-726bf5ed7e18\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jpzmn" Jul 23 04:40:45.234600 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.234572 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3d7f6ab8-9a28-4631-a93c-726bf5ed7e18-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-jpzmn\" (UID: \"3d7f6ab8-9a28-4631-a93c-726bf5ed7e18\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jpzmn" Jul 23 04:40:45.234816 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.234607 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3d7f6ab8-9a28-4631-a93c-726bf5ed7e18-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-jpzmn\" (UID: \"3d7f6ab8-9a28-4631-a93c-726bf5ed7e18\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jpzmn" Jul 23 04:40:45.234816 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.234635 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3d7f6ab8-9a28-4631-a93c-726bf5ed7e18-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-jpzmn\" (UID: \"3d7f6ab8-9a28-4631-a93c-726bf5ed7e18\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jpzmn" Jul 23 04:40:45.234816 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:45.234743 2562 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 23 04:40:45.234816 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:45.234818 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d7f6ab8-9a28-4631-a93c-726bf5ed7e18-prometheus-operator-tls podName:3d7f6ab8-9a28-4631-a93c-726bf5ed7e18 nodeName:}" failed. No retries permitted until 2026-07-23 04:40:45.734798661 +0000 UTC m=+81.077311550 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/3d7f6ab8-9a28-4631-a93c-726bf5ed7e18-prometheus-operator-tls") pod "prometheus-operator-5945b5cfd-jpzmn" (UID: "3d7f6ab8-9a28-4631-a93c-726bf5ed7e18") : secret "prometheus-operator-tls" not found Jul 23 04:40:45.235453 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.235419 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3d7f6ab8-9a28-4631-a93c-726bf5ed7e18-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-jpzmn\" (UID: \"3d7f6ab8-9a28-4631-a93c-726bf5ed7e18\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jpzmn" Jul 23 04:40:45.237416 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.237390 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3d7f6ab8-9a28-4631-a93c-726bf5ed7e18-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-jpzmn\" (UID: \"3d7f6ab8-9a28-4631-a93c-726bf5ed7e18\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jpzmn" Jul 23 04:40:45.245527 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.245502 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vqpr\" (UniqueName: \"kubernetes.io/projected/3d7f6ab8-9a28-4631-a93c-726bf5ed7e18-kube-api-access-4vqpr\") pod \"prometheus-operator-5945b5cfd-jpzmn\" (UID: \"3d7f6ab8-9a28-4631-a93c-726bf5ed7e18\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jpzmn" Jul 23 04:40:45.354206 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.354175 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7675f5469-nwfsm"] Jul 23 04:40:45.357924 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.357573 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7675f5469-nwfsm" Jul 23 04:40:45.361580 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.361556 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 23 04:40:45.361705 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.361582 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 23 04:40:45.365457 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.364925 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-25b8x\"" Jul 23 04:40:45.365457 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.364957 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 23 04:40:45.365457 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.365190 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 23 04:40:45.365457 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.365248 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 23 04:40:45.374586 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.374562 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7675f5469-nwfsm"] Jul 23 04:40:45.536787 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.536706 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c347ce4-da6a-4e19-91c9-ac29be9cef57-service-ca\") pod \"console-7675f5469-nwfsm\" (UID: \"3c347ce4-da6a-4e19-91c9-ac29be9cef57\") " pod="openshift-console/console-7675f5469-nwfsm" Jul 23 04:40:45.536787 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.536753 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjhcm\" (UniqueName: \"kubernetes.io/projected/3c347ce4-da6a-4e19-91c9-ac29be9cef57-kube-api-access-gjhcm\") pod \"console-7675f5469-nwfsm\" (UID: \"3c347ce4-da6a-4e19-91c9-ac29be9cef57\") " pod="openshift-console/console-7675f5469-nwfsm" Jul 23 04:40:45.536787 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.536774 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c347ce4-da6a-4e19-91c9-ac29be9cef57-console-config\") pod \"console-7675f5469-nwfsm\" (UID: \"3c347ce4-da6a-4e19-91c9-ac29be9cef57\") " pod="openshift-console/console-7675f5469-nwfsm" Jul 23 04:40:45.537050 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.536851 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c347ce4-da6a-4e19-91c9-ac29be9cef57-console-oauth-config\") pod \"console-7675f5469-nwfsm\" (UID: \"3c347ce4-da6a-4e19-91c9-ac29be9cef57\") " pod="openshift-console/console-7675f5469-nwfsm" Jul 23 04:40:45.537050 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.536882 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c347ce4-da6a-4e19-91c9-ac29be9cef57-console-serving-cert\") pod \"console-7675f5469-nwfsm\" (UID: \"3c347ce4-da6a-4e19-91c9-ac29be9cef57\") " pod="openshift-console/console-7675f5469-nwfsm" Jul 23 04:40:45.537050 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.536906 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c347ce4-da6a-4e19-91c9-ac29be9cef57-oauth-serving-cert\") pod \"console-7675f5469-nwfsm\" (UID: \"3c347ce4-da6a-4e19-91c9-ac29be9cef57\") " pod="openshift-console/console-7675f5469-nwfsm" Jul 23 04:40:45.638049 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.638012 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c347ce4-da6a-4e19-91c9-ac29be9cef57-service-ca\") pod \"console-7675f5469-nwfsm\" (UID: \"3c347ce4-da6a-4e19-91c9-ac29be9cef57\") " pod="openshift-console/console-7675f5469-nwfsm" Jul 23 04:40:45.638049 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.638053 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjhcm\" (UniqueName: \"kubernetes.io/projected/3c347ce4-da6a-4e19-91c9-ac29be9cef57-kube-api-access-gjhcm\") pod \"console-7675f5469-nwfsm\" (UID: \"3c347ce4-da6a-4e19-91c9-ac29be9cef57\") " pod="openshift-console/console-7675f5469-nwfsm" Jul 23 04:40:45.638302 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.638073 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c347ce4-da6a-4e19-91c9-ac29be9cef57-console-config\") pod \"console-7675f5469-nwfsm\" (UID: \"3c347ce4-da6a-4e19-91c9-ac29be9cef57\") " pod="openshift-console/console-7675f5469-nwfsm" Jul 23 04:40:45.638302 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.638211 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c347ce4-da6a-4e19-91c9-ac29be9cef57-console-oauth-config\") pod \"console-7675f5469-nwfsm\" (UID: \"3c347ce4-da6a-4e19-91c9-ac29be9cef57\") " pod="openshift-console/console-7675f5469-nwfsm" Jul 23 04:40:45.638302 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.638257 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c347ce4-da6a-4e19-91c9-ac29be9cef57-console-serving-cert\") pod \"console-7675f5469-nwfsm\" (UID: \"3c347ce4-da6a-4e19-91c9-ac29be9cef57\") " pod="openshift-console/console-7675f5469-nwfsm" Jul 23 04:40:45.638302 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.638284 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c347ce4-da6a-4e19-91c9-ac29be9cef57-oauth-serving-cert\") pod \"console-7675f5469-nwfsm\" (UID: \"3c347ce4-da6a-4e19-91c9-ac29be9cef57\") " pod="openshift-console/console-7675f5469-nwfsm" Jul 23 04:40:45.639010 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.638862 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c347ce4-da6a-4e19-91c9-ac29be9cef57-service-ca\") pod \"console-7675f5469-nwfsm\" (UID: \"3c347ce4-da6a-4e19-91c9-ac29be9cef57\") " pod="openshift-console/console-7675f5469-nwfsm" Jul 23 04:40:45.639134 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.639056 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c347ce4-da6a-4e19-91c9-ac29be9cef57-oauth-serving-cert\") pod \"console-7675f5469-nwfsm\" (UID: \"3c347ce4-da6a-4e19-91c9-ac29be9cef57\") " pod="openshift-console/console-7675f5469-nwfsm" Jul 23 04:40:45.639134 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.639068 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c347ce4-da6a-4e19-91c9-ac29be9cef57-console-config\") pod \"console-7675f5469-nwfsm\" (UID: \"3c347ce4-da6a-4e19-91c9-ac29be9cef57\") " pod="openshift-console/console-7675f5469-nwfsm" Jul 23 04:40:45.641359 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.641334 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c347ce4-da6a-4e19-91c9-ac29be9cef57-console-oauth-config\") pod \"console-7675f5469-nwfsm\" (UID: \"3c347ce4-da6a-4e19-91c9-ac29be9cef57\") " pod="openshift-console/console-7675f5469-nwfsm" Jul 23 04:40:45.641359 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.641349 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c347ce4-da6a-4e19-91c9-ac29be9cef57-console-serving-cert\") pod \"console-7675f5469-nwfsm\" (UID: \"3c347ce4-da6a-4e19-91c9-ac29be9cef57\") " pod="openshift-console/console-7675f5469-nwfsm" Jul 23 04:40:45.652117 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.652096 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjhcm\" (UniqueName: \"kubernetes.io/projected/3c347ce4-da6a-4e19-91c9-ac29be9cef57-kube-api-access-gjhcm\") pod \"console-7675f5469-nwfsm\" (UID: \"3c347ce4-da6a-4e19-91c9-ac29be9cef57\") " pod="openshift-console/console-7675f5469-nwfsm" Jul 23 04:40:45.672965 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.672935 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7675f5469-nwfsm" Jul 23 04:40:45.739283 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.739241 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3d7f6ab8-9a28-4631-a93c-726bf5ed7e18-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-jpzmn\" (UID: \"3d7f6ab8-9a28-4631-a93c-726bf5ed7e18\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jpzmn" Jul 23 04:40:45.742230 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.742204 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3d7f6ab8-9a28-4631-a93c-726bf5ed7e18-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-jpzmn\" (UID: \"3d7f6ab8-9a28-4631-a93c-726bf5ed7e18\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jpzmn" Jul 23 04:40:45.859617 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:45.859574 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-jpzmn" Jul 23 04:40:50.606594 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:50.606569 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7675f5469-nwfsm"] Jul 23 04:40:50.627002 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:50.626981 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-jpzmn"] Jul 23 04:40:50.703252 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:40:50.703218 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c347ce4_da6a_4e19_91c9_ac29be9cef57.slice/crio-04aaf63018f159891d8620585fd459a10713821edce0911ba6ddcbada5511704 WatchSource:0}: Error finding container 04aaf63018f159891d8620585fd459a10713821edce0911ba6ddcbada5511704: Status 404 returned error can't find the container with id 04aaf63018f159891d8620585fd459a10713821edce0911ba6ddcbada5511704 Jul 23 04:40:50.703812 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:40:50.703783 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d7f6ab8_9a28_4631_a93c_726bf5ed7e18.slice/crio-bf6bbb9d5ecf7cce96aa6faf51dbcc11e87ffc674368d92c26d4c9b7449b2885 WatchSource:0}: Error finding container bf6bbb9d5ecf7cce96aa6faf51dbcc11e87ffc674368d92c26d4c9b7449b2885: Status 404 returned error can't find the container with id bf6bbb9d5ecf7cce96aa6faf51dbcc11e87ffc674368d92c26d4c9b7449b2885 Jul 23 04:40:51.613667 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:51.613602 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-pzvsl" event={"ID":"1a3d4f1d-65d6-4abd-a649-6353217d4c77","Type":"ContainerStarted","Data":"120a87a51ec36600a7aa4accd4225bd78b4474bc6bd258b5b12218008a7c77d1"} Jul 23 04:40:51.614699 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:51.614638 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-pzvsl" Jul 23 04:40:51.617164 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:51.617139 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7675f5469-nwfsm" event={"ID":"3c347ce4-da6a-4e19-91c9-ac29be9cef57","Type":"ContainerStarted","Data":"04aaf63018f159891d8620585fd459a10713821edce0911ba6ddcbada5511704"} Jul 23 04:40:51.619039 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:51.618881 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-jpzmn" event={"ID":"3d7f6ab8-9a28-4631-a93c-726bf5ed7e18","Type":"ContainerStarted","Data":"bf6bbb9d5ecf7cce96aa6faf51dbcc11e87ffc674368d92c26d4c9b7449b2885"} Jul 23 04:40:51.629580 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:51.629526 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-pzvsl" Jul 23 04:40:51.634957 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:51.634910 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-pzvsl" podStartSLOduration=2.075273994 podStartE2EDuration="19.634891269s" totalImagesPullingTime="17.559617275s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:40:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:40:51.633454786 +0000 UTC m=+86.975967679" watchObservedRunningTime="2026-07-23 04:40:51.634891269 +0000 UTC m=+86.977404160" Jul 23 04:40:54.633545 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:54.633512 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-jpzmn" event={"ID":"3d7f6ab8-9a28-4631-a93c-726bf5ed7e18","Type":"ContainerStarted","Data":"df0feb3a246a343638c4ab09805dfc3e93d35562bbd639f00d7c65ed6e8675b8"} Jul 23 04:40:55.545789 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:55.545762 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5f446fbc9d-hzvct" Jul 23 04:40:55.639197 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:55.639161 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7675f5469-nwfsm" event={"ID":"3c347ce4-da6a-4e19-91c9-ac29be9cef57","Type":"ContainerStarted","Data":"549c82235838023178c30ffa55cc21889626b78bcbb87e1d797c0fa73bd2a97e"} Jul 23 04:40:55.641347 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:55.641315 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-jpzmn" event={"ID":"3d7f6ab8-9a28-4631-a93c-726bf5ed7e18","Type":"ContainerStarted","Data":"4fab82f92661e0679a1fe8017c6cbe489bf80a0bc2639b9f37033300d31b9bb9"} Jul 23 04:40:55.668232 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:55.663873 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7675f5469-nwfsm" podStartSLOduration=6.666234154 podStartE2EDuration="10.663852885s" totalImagesPullingTime="3.997618731s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:40:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:40:55.661777626 +0000 UTC m=+91.004290515" watchObservedRunningTime="2026-07-23 04:40:55.663852885 +0000 UTC m=+91.006365771" Jul 23 04:40:55.674065 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:55.674036 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7675f5469-nwfsm" Jul 23 04:40:55.674180 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:55.674110 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7675f5469-nwfsm" Jul 23 04:40:55.679690 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:55.679663 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7675f5469-nwfsm" Jul 23 04:40:55.692567 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:55.692517 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-jpzmn" podStartSLOduration=7.973399633 podStartE2EDuration="11.69250401s" totalImagesPullingTime="3.719104377s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:40:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:40:55.691580965 +0000 UTC m=+91.034093854" watchObservedRunningTime="2026-07-23 04:40:55.69250401 +0000 UTC m=+91.035016898" Jul 23 04:40:56.650324 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:56.650289 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7675f5469-nwfsm" Jul 23 04:40:57.339287 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.339250 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-tk424"] Jul 23 04:40:57.399684 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.399645 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-tk424"] Jul 23 04:40:57.399881 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.399701 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-m5fv5"] Jul 23 04:40:57.399881 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.399796 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-tk424" Jul 23 04:40:57.402675 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.402648 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 23 04:40:57.402981 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.402956 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 23 04:40:57.403209 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.403164 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-vm7m9\"" Jul 23 04:40:57.429851 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.429813 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-m5fv5"] Jul 23 04:40:57.429952 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.429891 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6pnvh"] Jul 23 04:40:57.430042 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.430025 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-m5fv5" Jul 23 04:40:57.432900 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.432879 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 23 04:40:57.433034 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.433016 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 23 04:40:57.433114 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.432900 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-lgjdc\"" Jul 23 04:40:57.433195 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.432899 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 23 04:40:57.443146 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.443116 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3809e4cf-5052-4402-ada4-7668bdf98a74-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-m5fv5\" (UID: \"3809e4cf-5052-4402-ada4-7668bdf98a74\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-m5fv5" Jul 23 04:40:57.443257 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.443184 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3809e4cf-5052-4402-ada4-7668bdf98a74-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-m5fv5\" (UID: \"3809e4cf-5052-4402-ada4-7668bdf98a74\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-m5fv5" Jul 23 04:40:57.443257 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.443213 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3809e4cf-5052-4402-ada4-7668bdf98a74-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-m5fv5\" (UID: \"3809e4cf-5052-4402-ada4-7668bdf98a74\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-m5fv5" Jul 23 04:40:57.443257 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.443245 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/49aa1a49-bb25-47e7-9f6c-686e559571f7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-tk424\" (UID: \"49aa1a49-bb25-47e7-9f6c-686e559571f7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-tk424" Jul 23 04:40:57.443425 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.443281 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hh8v\" (UniqueName: \"kubernetes.io/projected/3809e4cf-5052-4402-ada4-7668bdf98a74-kube-api-access-9hh8v\") pod \"kube-state-metrics-56c5594674-m5fv5\" (UID: \"3809e4cf-5052-4402-ada4-7668bdf98a74\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-m5fv5" Jul 23 04:40:57.443425 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.443315 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/49aa1a49-bb25-47e7-9f6c-686e559571f7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-tk424\" (UID: \"49aa1a49-bb25-47e7-9f6c-686e559571f7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-tk424" Jul 23 04:40:57.443425 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.443362 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49aa1a49-bb25-47e7-9f6c-686e559571f7-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-tk424\" (UID: \"49aa1a49-bb25-47e7-9f6c-686e559571f7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-tk424" Jul 23 04:40:57.443425 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.443386 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3809e4cf-5052-4402-ada4-7668bdf98a74-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-m5fv5\" (UID: \"3809e4cf-5052-4402-ada4-7668bdf98a74\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-m5fv5" Jul 23 04:40:57.443668 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.443473 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3809e4cf-5052-4402-ada4-7668bdf98a74-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-m5fv5\" (UID: \"3809e4cf-5052-4402-ada4-7668bdf98a74\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-m5fv5" Jul 23 04:40:57.443668 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.443509 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7phwf\" (UniqueName: \"kubernetes.io/projected/49aa1a49-bb25-47e7-9f6c-686e559571f7-kube-api-access-7phwf\") pod \"openshift-state-metrics-79f6958cdf-tk424\" (UID: \"49aa1a49-bb25-47e7-9f6c-686e559571f7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-tk424" Jul 23 04:40:57.451977 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.451951 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6pnvh" Jul 23 04:40:57.454598 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.454573 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-q7zxn\"" Jul 23 04:40:57.454877 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.454578 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 23 04:40:57.455021 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.454586 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 23 04:40:57.455162 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.454620 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 23 04:40:57.544719 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.544675 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3809e4cf-5052-4402-ada4-7668bdf98a74-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-m5fv5\" (UID: \"3809e4cf-5052-4402-ada4-7668bdf98a74\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-m5fv5" Jul 23 04:40:57.544984 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.544964 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7phwf\" (UniqueName: \"kubernetes.io/projected/49aa1a49-bb25-47e7-9f6c-686e559571f7-kube-api-access-7phwf\") pod \"openshift-state-metrics-79f6958cdf-tk424\" (UID: \"49aa1a49-bb25-47e7-9f6c-686e559571f7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-tk424" Jul 23 04:40:57.545126 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.545109 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc-node-exporter-wtmp\") pod \"node-exporter-6pnvh\" (UID: \"a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc\") " pod="openshift-monitoring/node-exporter-6pnvh" Jul 23 04:40:57.545238 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.545222 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc-sys\") pod \"node-exporter-6pnvh\" (UID: \"a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc\") " pod="openshift-monitoring/node-exporter-6pnvh" Jul 23 04:40:57.545401 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.545371 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3809e4cf-5052-4402-ada4-7668bdf98a74-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-m5fv5\" (UID: \"3809e4cf-5052-4402-ada4-7668bdf98a74\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-m5fv5" Jul 23 04:40:57.545480 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.545436 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc-node-exporter-textfile\") pod \"node-exporter-6pnvh\" (UID: \"a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc\") " pod="openshift-monitoring/node-exporter-6pnvh" Jul 23 04:40:57.545480 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.545469 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc-node-exporter-accelerators-collector-config\") pod \"node-exporter-6pnvh\" (UID: \"a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc\") " pod="openshift-monitoring/node-exporter-6pnvh" Jul 23 04:40:57.545588 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.545503 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6pnvh\" (UID: \"a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc\") " pod="openshift-monitoring/node-exporter-6pnvh" Jul 23 04:40:57.545588 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.545532 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3809e4cf-5052-4402-ada4-7668bdf98a74-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-m5fv5\" (UID: \"3809e4cf-5052-4402-ada4-7668bdf98a74\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-m5fv5" Jul 23 04:40:57.545588 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.545557 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3809e4cf-5052-4402-ada4-7668bdf98a74-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-m5fv5\" (UID: \"3809e4cf-5052-4402-ada4-7668bdf98a74\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-m5fv5" Jul 23 04:40:57.545734 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.545587 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/49aa1a49-bb25-47e7-9f6c-686e559571f7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-tk424\" (UID: \"49aa1a49-bb25-47e7-9f6c-686e559571f7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-tk424" Jul 23 04:40:57.545734 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.545618 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48jtd\" (UniqueName: \"kubernetes.io/projected/a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc-kube-api-access-48jtd\") pod \"node-exporter-6pnvh\" (UID: \"a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc\") " pod="openshift-monitoring/node-exporter-6pnvh" Jul 23 04:40:57.545734 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.545646 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hh8v\" (UniqueName: \"kubernetes.io/projected/3809e4cf-5052-4402-ada4-7668bdf98a74-kube-api-access-9hh8v\") pod \"kube-state-metrics-56c5594674-m5fv5\" (UID: \"3809e4cf-5052-4402-ada4-7668bdf98a74\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-m5fv5" Jul 23 04:40:57.545734 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.545682 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/49aa1a49-bb25-47e7-9f6c-686e559571f7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-tk424\" (UID: \"49aa1a49-bb25-47e7-9f6c-686e559571f7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-tk424" Jul 23 04:40:57.545734 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.545703 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc-root\") pod \"node-exporter-6pnvh\" (UID: \"a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc\") " pod="openshift-monitoring/node-exporter-6pnvh" Jul 23 04:40:57.545734 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.545730 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc-metrics-client-ca\") pod \"node-exporter-6pnvh\" (UID: \"a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc\") " pod="openshift-monitoring/node-exporter-6pnvh" Jul 23 04:40:57.546061 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.545767 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc-node-exporter-tls\") pod \"node-exporter-6pnvh\" (UID: \"a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc\") " pod="openshift-monitoring/node-exporter-6pnvh" Jul 23 04:40:57.546061 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.545790 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49aa1a49-bb25-47e7-9f6c-686e559571f7-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-tk424\" (UID: \"49aa1a49-bb25-47e7-9f6c-686e559571f7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-tk424" Jul 23 04:40:57.546061 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.545815 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3809e4cf-5052-4402-ada4-7668bdf98a74-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-m5fv5\" (UID: \"3809e4cf-5052-4402-ada4-7668bdf98a74\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-m5fv5" Jul 23 04:40:57.546216 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.546164 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3809e4cf-5052-4402-ada4-7668bdf98a74-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-m5fv5\" (UID: \"3809e4cf-5052-4402-ada4-7668bdf98a74\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-m5fv5" Jul 23 04:40:57.546216 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.546188 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3809e4cf-5052-4402-ada4-7668bdf98a74-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-m5fv5\" (UID: \"3809e4cf-5052-4402-ada4-7668bdf98a74\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-m5fv5" Jul 23 04:40:57.546815 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.546787 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3809e4cf-5052-4402-ada4-7668bdf98a74-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-m5fv5\" (UID: \"3809e4cf-5052-4402-ada4-7668bdf98a74\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-m5fv5" Jul 23 04:40:57.547044 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:57.547024 2562 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 23 04:40:57.547203 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:57.547191 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3809e4cf-5052-4402-ada4-7668bdf98a74-kube-state-metrics-tls podName:3809e4cf-5052-4402-ada4-7668bdf98a74 nodeName:}" failed. No retries permitted until 2026-07-23 04:40:58.047171773 +0000 UTC m=+93.389684654 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/3809e4cf-5052-4402-ada4-7668bdf98a74-kube-state-metrics-tls") pod "kube-state-metrics-56c5594674-m5fv5" (UID: "3809e4cf-5052-4402-ada4-7668bdf98a74") : secret "kube-state-metrics-tls" not found Jul 23 04:40:57.547803 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.547620 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49aa1a49-bb25-47e7-9f6c-686e559571f7-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-tk424\" (UID: \"49aa1a49-bb25-47e7-9f6c-686e559571f7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-tk424" Jul 23 04:40:57.549823 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.549774 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/49aa1a49-bb25-47e7-9f6c-686e559571f7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-tk424\" (UID: \"49aa1a49-bb25-47e7-9f6c-686e559571f7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-tk424" Jul 23 04:40:57.549823 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.549813 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3809e4cf-5052-4402-ada4-7668bdf98a74-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-m5fv5\" (UID: \"3809e4cf-5052-4402-ada4-7668bdf98a74\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-m5fv5" Jul 23 04:40:57.551513 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.551491 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/49aa1a49-bb25-47e7-9f6c-686e559571f7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-tk424\" (UID: \"49aa1a49-bb25-47e7-9f6c-686e559571f7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-tk424" Jul 23 04:40:57.556606 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.556569 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7phwf\" (UniqueName: \"kubernetes.io/projected/49aa1a49-bb25-47e7-9f6c-686e559571f7-kube-api-access-7phwf\") pod \"openshift-state-metrics-79f6958cdf-tk424\" (UID: \"49aa1a49-bb25-47e7-9f6c-686e559571f7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-tk424" Jul 23 04:40:57.568270 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.568243 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hh8v\" (UniqueName: \"kubernetes.io/projected/3809e4cf-5052-4402-ada4-7668bdf98a74-kube-api-access-9hh8v\") pod \"kube-state-metrics-56c5594674-m5fv5\" (UID: \"3809e4cf-5052-4402-ada4-7668bdf98a74\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-m5fv5" Jul 23 04:40:57.647075 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.646992 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc-node-exporter-wtmp\") pod \"node-exporter-6pnvh\" (UID: \"a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc\") " pod="openshift-monitoring/node-exporter-6pnvh" Jul 23 04:40:57.647075 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.647040 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc-sys\") pod \"node-exporter-6pnvh\" (UID: \"a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc\") " pod="openshift-monitoring/node-exporter-6pnvh" Jul 23 04:40:57.647292 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.647091 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc-node-exporter-textfile\") pod \"node-exporter-6pnvh\" (UID: \"a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc\") " pod="openshift-monitoring/node-exporter-6pnvh" Jul 23 04:40:57.647292 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.647120 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc-node-exporter-accelerators-collector-config\") pod \"node-exporter-6pnvh\" (UID: \"a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc\") " pod="openshift-monitoring/node-exporter-6pnvh" Jul 23 04:40:57.647292 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.647143 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6pnvh\" (UID: \"a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc\") " pod="openshift-monitoring/node-exporter-6pnvh" Jul 23 04:40:57.647292 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.647170 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc-node-exporter-wtmp\") pod \"node-exporter-6pnvh\" (UID: \"a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc\") " pod="openshift-monitoring/node-exporter-6pnvh" Jul 23 04:40:57.647292 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.647189 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48jtd\" (UniqueName: \"kubernetes.io/projected/a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc-kube-api-access-48jtd\") pod \"node-exporter-6pnvh\" (UID: \"a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc\") " pod="openshift-monitoring/node-exporter-6pnvh" Jul 23 04:40:57.647292 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.647170 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc-sys\") pod \"node-exporter-6pnvh\" (UID: \"a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc\") " pod="openshift-monitoring/node-exporter-6pnvh" Jul 23 04:40:57.647292 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.647248 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc-root\") pod \"node-exporter-6pnvh\" (UID: \"a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc\") " pod="openshift-monitoring/node-exporter-6pnvh" Jul 23 04:40:57.647292 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.647279 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc-metrics-client-ca\") pod \"node-exporter-6pnvh\" (UID: \"a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc\") " pod="openshift-monitoring/node-exporter-6pnvh" Jul 23 04:40:57.647666 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.647319 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc-node-exporter-tls\") pod \"node-exporter-6pnvh\" (UID: \"a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc\") " pod="openshift-monitoring/node-exporter-6pnvh" Jul 23 04:40:57.647866 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.647820 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc-node-exporter-accelerators-collector-config\") pod \"node-exporter-6pnvh\" (UID: \"a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc\") " pod="openshift-monitoring/node-exporter-6pnvh" Jul 23 04:40:57.648173 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.648094 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc-node-exporter-textfile\") pod \"node-exporter-6pnvh\" (UID: \"a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc\") " pod="openshift-monitoring/node-exporter-6pnvh" Jul 23 04:40:57.649261 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.649058 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc-root\") pod \"node-exporter-6pnvh\" (UID: \"a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc\") " pod="openshift-monitoring/node-exporter-6pnvh" Jul 23 04:40:57.650546 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.649553 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc-metrics-client-ca\") pod \"node-exporter-6pnvh\" (UID: \"a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc\") " pod="openshift-monitoring/node-exporter-6pnvh" Jul 23 04:40:57.650546 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.650241 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6pnvh\" (UID: \"a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc\") " pod="openshift-monitoring/node-exporter-6pnvh" Jul 23 04:40:57.650546 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.650306 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc-node-exporter-tls\") pod \"node-exporter-6pnvh\" (UID: \"a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc\") " pod="openshift-monitoring/node-exporter-6pnvh" Jul 23 04:40:57.662328 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.662306 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48jtd\" (UniqueName: \"kubernetes.io/projected/a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc-kube-api-access-48jtd\") pod \"node-exporter-6pnvh\" (UID: \"a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc\") " pod="openshift-monitoring/node-exporter-6pnvh" Jul 23 04:40:57.711343 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.711310 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-tk424" Jul 23 04:40:57.713057 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.713018 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" podUID="d15db39a-c63b-4806-86a1-ba472f599e54" containerName="registry" containerID="cri-o://33f7fdcbb91ec750405439c4a033aadcefd6f3b5087e109aad11ca28c69b88ec" gracePeriod=30 Jul 23 04:40:57.761995 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.761964 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6pnvh" Jul 23 04:40:57.907920 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:57.907813 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-tk424"] Jul 23 04:40:57.910491 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:40:57.910457 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49aa1a49_bb25_47e7_9f6c_686e559571f7.slice/crio-767e01170805fe81bb5875bf9383a02f12dbba0dd7fae65fdc6fddb0b4ca4874 WatchSource:0}: Error finding container 767e01170805fe81bb5875bf9383a02f12dbba0dd7fae65fdc6fddb0b4ca4874: Status 404 returned error can't find the container with id 767e01170805fe81bb5875bf9383a02f12dbba0dd7fae65fdc6fddb0b4ca4874 Jul 23 04:40:58.033906 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.033882 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" Jul 23 04:40:58.050644 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.050611 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vfk59\" (UniqueName: \"kubernetes.io/projected/d15db39a-c63b-4806-86a1-ba472f599e54-kube-api-access-vfk59\") pod \"d15db39a-c63b-4806-86a1-ba472f599e54\" (UID: \"d15db39a-c63b-4806-86a1-ba472f599e54\") " Jul 23 04:40:58.050809 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.050668 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d15db39a-c63b-4806-86a1-ba472f599e54-registry-tls\") pod \"d15db39a-c63b-4806-86a1-ba472f599e54\" (UID: \"d15db39a-c63b-4806-86a1-ba472f599e54\") " Jul 23 04:40:58.050809 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.050702 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d15db39a-c63b-4806-86a1-ba472f599e54-trusted-ca\") pod \"d15db39a-c63b-4806-86a1-ba472f599e54\" (UID: \"d15db39a-c63b-4806-86a1-ba472f599e54\") " Jul 23 04:40:58.050809 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.050736 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d15db39a-c63b-4806-86a1-ba472f599e54-image-registry-private-configuration\") pod \"d15db39a-c63b-4806-86a1-ba472f599e54\" (UID: \"d15db39a-c63b-4806-86a1-ba472f599e54\") " Jul 23 04:40:58.050809 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.050767 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d15db39a-c63b-4806-86a1-ba472f599e54-ca-trust-extracted\") pod \"d15db39a-c63b-4806-86a1-ba472f599e54\" (UID: \"d15db39a-c63b-4806-86a1-ba472f599e54\") " Jul 23 04:40:58.050809 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.050807 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d15db39a-c63b-4806-86a1-ba472f599e54-registry-certificates\") pod \"d15db39a-c63b-4806-86a1-ba472f599e54\" (UID: \"d15db39a-c63b-4806-86a1-ba472f599e54\") " Jul 23 04:40:58.051085 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.050882 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d15db39a-c63b-4806-86a1-ba472f599e54-installation-pull-secrets\") pod \"d15db39a-c63b-4806-86a1-ba472f599e54\" (UID: \"d15db39a-c63b-4806-86a1-ba472f599e54\") " Jul 23 04:40:58.051085 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.050921 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d15db39a-c63b-4806-86a1-ba472f599e54-bound-sa-token\") pod \"d15db39a-c63b-4806-86a1-ba472f599e54\" (UID: \"d15db39a-c63b-4806-86a1-ba472f599e54\") " Jul 23 04:40:58.051746 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.051685 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d15db39a-c63b-4806-86a1-ba472f599e54-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "d15db39a-c63b-4806-86a1-ba472f599e54" (UID: "d15db39a-c63b-4806-86a1-ba472f599e54"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 04:40:58.052238 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.052001 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d15db39a-c63b-4806-86a1-ba472f599e54-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "d15db39a-c63b-4806-86a1-ba472f599e54" (UID: "d15db39a-c63b-4806-86a1-ba472f599e54"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 04:40:58.059059 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.058981 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3809e4cf-5052-4402-ada4-7668bdf98a74-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-m5fv5\" (UID: \"3809e4cf-5052-4402-ada4-7668bdf98a74\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-m5fv5" Jul 23 04:40:58.059197 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.059062 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d15db39a-c63b-4806-86a1-ba472f599e54-trusted-ca\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:40:58.059197 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.059089 2562 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d15db39a-c63b-4806-86a1-ba472f599e54-registry-certificates\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:40:58.060564 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.059496 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d15db39a-c63b-4806-86a1-ba472f599e54-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "d15db39a-c63b-4806-86a1-ba472f599e54" (UID: "d15db39a-c63b-4806-86a1-ba472f599e54"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 04:40:58.060564 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.059617 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d15db39a-c63b-4806-86a1-ba472f599e54-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "d15db39a-c63b-4806-86a1-ba472f599e54" (UID: "d15db39a-c63b-4806-86a1-ba472f599e54"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 04:40:58.060564 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.059894 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d15db39a-c63b-4806-86a1-ba472f599e54-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d15db39a-c63b-4806-86a1-ba472f599e54" (UID: "d15db39a-c63b-4806-86a1-ba472f599e54"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 04:40:58.060792 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.060616 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d15db39a-c63b-4806-86a1-ba472f599e54-kube-api-access-vfk59" (OuterVolumeSpecName: "kube-api-access-vfk59") pod "d15db39a-c63b-4806-86a1-ba472f599e54" (UID: "d15db39a-c63b-4806-86a1-ba472f599e54"). InnerVolumeSpecName "kube-api-access-vfk59". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 04:40:58.063480 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.062074 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3809e4cf-5052-4402-ada4-7668bdf98a74-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-m5fv5\" (UID: \"3809e4cf-5052-4402-ada4-7668bdf98a74\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-m5fv5" Jul 23 04:40:58.063480 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.061559 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d15db39a-c63b-4806-86a1-ba472f599e54-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "d15db39a-c63b-4806-86a1-ba472f599e54" (UID: "d15db39a-c63b-4806-86a1-ba472f599e54"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 04:40:58.072285 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.072215 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d15db39a-c63b-4806-86a1-ba472f599e54-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "d15db39a-c63b-4806-86a1-ba472f599e54" (UID: "d15db39a-c63b-4806-86a1-ba472f599e54"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 04:40:58.160058 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.159988 2562 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d15db39a-c63b-4806-86a1-ba472f599e54-ca-trust-extracted\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:40:58.160058 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.160022 2562 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d15db39a-c63b-4806-86a1-ba472f599e54-installation-pull-secrets\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:40:58.160058 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.160038 2562 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d15db39a-c63b-4806-86a1-ba472f599e54-bound-sa-token\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:40:58.160058 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.160051 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfk59\" (UniqueName: \"kubernetes.io/projected/d15db39a-c63b-4806-86a1-ba472f599e54-kube-api-access-vfk59\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:40:58.160340 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.160067 2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d15db39a-c63b-4806-86a1-ba472f599e54-registry-tls\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:40:58.160340 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.160084 2562 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d15db39a-c63b-4806-86a1-ba472f599e54-image-registry-private-configuration\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:40:58.344373 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.344333 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-m5fv5" Jul 23 04:40:58.410542 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.409812 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 23 04:40:58.410542 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.410109 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d15db39a-c63b-4806-86a1-ba472f599e54" containerName="registry" Jul 23 04:40:58.410542 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.410121 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d15db39a-c63b-4806-86a1-ba472f599e54" containerName="registry" Jul 23 04:40:58.410542 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.410160 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d15db39a-c63b-4806-86a1-ba472f599e54" containerName="registry" Jul 23 04:40:58.434859 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.432944 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.439852 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.438105 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 23 04:40:58.439852 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.438668 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-q5pv2\"" Jul 23 04:40:58.449159 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.445415 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 23 04:40:58.449159 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.445677 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 23 04:40:58.449159 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.445891 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 23 04:40:58.449159 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.446109 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 23 04:40:58.449159 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.446304 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 23 04:40:58.449159 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.446518 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 23 04:40:58.449159 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.446787 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 23 04:40:58.449159 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.446939 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 23 04:40:58.449159 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.447210 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 23 04:40:58.464253 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.463646 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.464253 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.463693 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.464253 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.463750 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.464253 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.463781 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-web-config\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.464253 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.463802 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.464253 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.463858 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.464253 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.463896 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-config-out\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.464253 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.463928 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.464253 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.463958 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smhjq\" (UniqueName: \"kubernetes.io/projected/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-kube-api-access-smhjq\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.464253 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.463984 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-config-volume\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.464253 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.464021 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.464253 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.464049 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.464253 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.464076 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.552676 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.552639 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-m5fv5"] Jul 23 04:40:58.560377 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:40:58.560341 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3809e4cf_5052_4402_ada4_7668bdf98a74.slice/crio-61ef077d633c09897bf52b9889eabc62bb122e4724e2e33f8bac193d753bbf7f WatchSource:0}: Error finding container 61ef077d633c09897bf52b9889eabc62bb122e4724e2e33f8bac193d753bbf7f: Status 404 returned error can't find the container with id 61ef077d633c09897bf52b9889eabc62bb122e4724e2e33f8bac193d753bbf7f Jul 23 04:40:58.564885 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.564815 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.564885 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.564880 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-web-config\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.565110 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.564904 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.565110 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.564940 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.565110 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.564974 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-config-out\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.565110 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.564998 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.565110 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.565023 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smhjq\" (UniqueName: \"kubernetes.io/projected/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-kube-api-access-smhjq\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.565110 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.565091 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-config-volume\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.565397 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.565140 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.565397 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.565169 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.565397 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.565196 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.565397 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.565244 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.565397 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.565271 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.568556 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.567645 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.568556 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.568004 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.568556 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.568267 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.569219 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.569123 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.570222 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.570173 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.571603 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.571572 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.574088 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.574035 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-config-out\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.576546 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.576505 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smhjq\" (UniqueName: \"kubernetes.io/projected/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-kube-api-access-smhjq\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.580012 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.579373 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-config-volume\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.580012 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.579771 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.580012 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.579777 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.580677 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.580176 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-web-config\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.582152 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.581947 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:58.656760 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.656723 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-m5fv5" event={"ID":"3809e4cf-5052-4402-ada4-7668bdf98a74","Type":"ContainerStarted","Data":"61ef077d633c09897bf52b9889eabc62bb122e4724e2e33f8bac193d753bbf7f"} Jul 23 04:40:58.658254 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.658221 2562 generic.go:356] "Generic (PLEG): container finished" podID="d15db39a-c63b-4806-86a1-ba472f599e54" containerID="33f7fdcbb91ec750405439c4a033aadcefd6f3b5087e109aad11ca28c69b88ec" exitCode=0 Jul 23 04:40:58.658402 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.658295 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" Jul 23 04:40:58.658402 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.658312 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" event={"ID":"d15db39a-c63b-4806-86a1-ba472f599e54","Type":"ContainerDied","Data":"33f7fdcbb91ec750405439c4a033aadcefd6f3b5087e109aad11ca28c69b88ec"} Jul 23 04:40:58.658402 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.658359 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6798b7cd97-rbnww" event={"ID":"d15db39a-c63b-4806-86a1-ba472f599e54","Type":"ContainerDied","Data":"deecfe25cf86443665ae0367f36132131593bc9b648039cf6f98975da4822ea0"} Jul 23 04:40:58.658402 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.658400 2562 scope.go:117] "RemoveContainer" containerID="33f7fdcbb91ec750405439c4a033aadcefd6f3b5087e109aad11ca28c69b88ec" Jul 23 04:40:58.659651 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.659625 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6pnvh" event={"ID":"a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc","Type":"ContainerStarted","Data":"cc43c8ec77137502ef22f936d9043b5a0fba2196339a5ca8b3866f64be658165"} Jul 23 04:40:58.666940 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.666870 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-tk424" event={"ID":"49aa1a49-bb25-47e7-9f6c-686e559571f7","Type":"ContainerStarted","Data":"13f8e5531febb3d4f3426eaf4f9c76f3aff9765db472b1e529d2aa21feabe204"} Jul 23 04:40:58.667061 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.666938 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-tk424" event={"ID":"49aa1a49-bb25-47e7-9f6c-686e559571f7","Type":"ContainerStarted","Data":"aa939d857741c4e101a5e0cd1a13aabd96dc6012d9477553be1854e60a1c47a5"} Jul 23 04:40:58.667061 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.666965 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-tk424" event={"ID":"49aa1a49-bb25-47e7-9f6c-686e559571f7","Type":"ContainerStarted","Data":"767e01170805fe81bb5875bf9383a02f12dbba0dd7fae65fdc6fddb0b4ca4874"} Jul 23 04:40:58.668261 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.668241 2562 scope.go:117] "RemoveContainer" containerID="33f7fdcbb91ec750405439c4a033aadcefd6f3b5087e109aad11ca28c69b88ec" Jul 23 04:40:58.669015 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:40:58.668919 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33f7fdcbb91ec750405439c4a033aadcefd6f3b5087e109aad11ca28c69b88ec\": container with ID starting with 33f7fdcbb91ec750405439c4a033aadcefd6f3b5087e109aad11ca28c69b88ec not found: ID does not exist" containerID="33f7fdcbb91ec750405439c4a033aadcefd6f3b5087e109aad11ca28c69b88ec" Jul 23 04:40:58.669015 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.668969 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33f7fdcbb91ec750405439c4a033aadcefd6f3b5087e109aad11ca28c69b88ec"} err="failed to get container status \"33f7fdcbb91ec750405439c4a033aadcefd6f3b5087e109aad11ca28c69b88ec\": rpc error: code = NotFound desc = could not find container \"33f7fdcbb91ec750405439c4a033aadcefd6f3b5087e109aad11ca28c69b88ec\": container with ID starting with 33f7fdcbb91ec750405439c4a033aadcefd6f3b5087e109aad11ca28c69b88ec not found: ID does not exist" Jul 23 04:40:58.683786 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.683752 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6798b7cd97-rbnww"] Jul 23 04:40:58.688618 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.688595 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6798b7cd97-rbnww"] Jul 23 04:40:58.764019 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:58.763790 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:40:59.198441 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:59.198406 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 23 04:40:59.211310 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:59.211257 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d15db39a-c63b-4806-86a1-ba472f599e54" path="/var/lib/kubelet/pods/d15db39a-c63b-4806-86a1-ba472f599e54/volumes" Jul 23 04:40:59.216442 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:40:59.216413 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd0f6674_bc91_4148_8b30_7bc7ae20f2cd.slice/crio-97cb5dc3fa67cc5f257e727d72ceeeb3054d7d221494385a0746eb40c3cab740 WatchSource:0}: Error finding container 97cb5dc3fa67cc5f257e727d72ceeeb3054d7d221494385a0746eb40c3cab740: Status 404 returned error can't find the container with id 97cb5dc3fa67cc5f257e727d72ceeeb3054d7d221494385a0746eb40c3cab740 Jul 23 04:40:59.677102 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:59.676997 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd","Type":"ContainerStarted","Data":"97cb5dc3fa67cc5f257e727d72ceeeb3054d7d221494385a0746eb40c3cab740"} Jul 23 04:40:59.782355 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:40:59.782248 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7675f5469-nwfsm"] Jul 23 04:41:00.365684 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:00.365638 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-765ff6dd64-cql22"] Jul 23 04:41:00.386943 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:00.386251 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-765ff6dd64-cql22"] Jul 23 04:41:00.386943 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:00.386347 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-765ff6dd64-cql22" Jul 23 04:41:00.390308 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:00.390280 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 23 04:41:00.390611 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:00.390593 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 23 04:41:00.390797 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:00.390782 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 23 04:41:00.391161 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:00.391145 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-4hcb8ql5h8cv7\"" Jul 23 04:41:00.391577 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:00.391558 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-wmm2m\"" Jul 23 04:41:00.392960 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:00.392937 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 23 04:41:00.393064 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:00.393039 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 23 04:41:00.489912 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:00.489877 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-765ff6dd64-cql22\" (UID: \"aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e\") " pod="openshift-monitoring/thanos-querier-765ff6dd64-cql22" Jul 23 04:41:00.489912 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:00.489914 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e-secret-thanos-querier-tls\") pod \"thanos-querier-765ff6dd64-cql22\" (UID: \"aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e\") " pod="openshift-monitoring/thanos-querier-765ff6dd64-cql22" Jul 23 04:41:00.490099 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:00.489962 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-765ff6dd64-cql22\" (UID: \"aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e\") " pod="openshift-monitoring/thanos-querier-765ff6dd64-cql22" Jul 23 04:41:00.490099 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:00.489985 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-765ff6dd64-cql22\" (UID: \"aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e\") " pod="openshift-monitoring/thanos-querier-765ff6dd64-cql22" Jul 23 04:41:00.490099 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:00.490007 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e-secret-grpc-tls\") pod \"thanos-querier-765ff6dd64-cql22\" (UID: \"aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e\") " pod="openshift-monitoring/thanos-querier-765ff6dd64-cql22" Jul 23 04:41:00.490099 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:00.490023 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e-metrics-client-ca\") pod \"thanos-querier-765ff6dd64-cql22\" (UID: \"aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e\") " pod="openshift-monitoring/thanos-querier-765ff6dd64-cql22" Jul 23 04:41:00.490099 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:00.490055 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-765ff6dd64-cql22\" (UID: \"aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e\") " pod="openshift-monitoring/thanos-querier-765ff6dd64-cql22" Jul 23 04:41:00.490099 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:00.490096 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pv5vb\" (UniqueName: \"kubernetes.io/projected/aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e-kube-api-access-pv5vb\") pod \"thanos-querier-765ff6dd64-cql22\" (UID: \"aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e\") " pod="openshift-monitoring/thanos-querier-765ff6dd64-cql22" Jul 23 04:41:00.591377 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:00.591345 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv5vb\" (UniqueName: \"kubernetes.io/projected/aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e-kube-api-access-pv5vb\") pod \"thanos-querier-765ff6dd64-cql22\" (UID: \"aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e\") " pod="openshift-monitoring/thanos-querier-765ff6dd64-cql22" Jul 23 04:41:00.591500 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:00.591446 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-765ff6dd64-cql22\" (UID: \"aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e\") " pod="openshift-monitoring/thanos-querier-765ff6dd64-cql22" Jul 23 04:41:00.591500 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:00.591485 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e-secret-thanos-querier-tls\") pod \"thanos-querier-765ff6dd64-cql22\" (UID: \"aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e\") " pod="openshift-monitoring/thanos-querier-765ff6dd64-cql22" Jul 23 04:41:00.591611 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:00.591540 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-765ff6dd64-cql22\" (UID: \"aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e\") " pod="openshift-monitoring/thanos-querier-765ff6dd64-cql22" Jul 23 04:41:00.591611 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:00.591562 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-765ff6dd64-cql22\" (UID: \"aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e\") " pod="openshift-monitoring/thanos-querier-765ff6dd64-cql22" Jul 23 04:41:00.591611 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:00.591590 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e-secret-grpc-tls\") pod \"thanos-querier-765ff6dd64-cql22\" (UID: \"aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e\") " pod="openshift-monitoring/thanos-querier-765ff6dd64-cql22" Jul 23 04:41:00.591761 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:00.591615 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e-metrics-client-ca\") pod \"thanos-querier-765ff6dd64-cql22\" (UID: \"aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e\") " pod="openshift-monitoring/thanos-querier-765ff6dd64-cql22" Jul 23 04:41:00.591761 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:00.591646 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-765ff6dd64-cql22\" (UID: \"aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e\") " pod="openshift-monitoring/thanos-querier-765ff6dd64-cql22" Jul 23 04:41:00.592758 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:00.592728 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e-metrics-client-ca\") pod \"thanos-querier-765ff6dd64-cql22\" (UID: \"aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e\") " pod="openshift-monitoring/thanos-querier-765ff6dd64-cql22" Jul 23 04:41:00.594881 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:00.594814 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-765ff6dd64-cql22\" (UID: \"aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e\") " pod="openshift-monitoring/thanos-querier-765ff6dd64-cql22" Jul 23 04:41:00.595499 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:00.595438 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e-secret-thanos-querier-tls\") pod \"thanos-querier-765ff6dd64-cql22\" (UID: \"aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e\") " pod="openshift-monitoring/thanos-querier-765ff6dd64-cql22" Jul 23 04:41:00.595499 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:00.595439 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-765ff6dd64-cql22\" (UID: \"aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e\") " pod="openshift-monitoring/thanos-querier-765ff6dd64-cql22" Jul 23 04:41:00.595635 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:00.595507 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-765ff6dd64-cql22\" (UID: \"aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e\") " pod="openshift-monitoring/thanos-querier-765ff6dd64-cql22" Jul 23 04:41:00.595635 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:00.595592 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-765ff6dd64-cql22\" (UID: \"aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e\") " pod="openshift-monitoring/thanos-querier-765ff6dd64-cql22" Jul 23 04:41:00.596088 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:00.596047 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e-secret-grpc-tls\") pod \"thanos-querier-765ff6dd64-cql22\" (UID: \"aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e\") " pod="openshift-monitoring/thanos-querier-765ff6dd64-cql22" Jul 23 04:41:00.603804 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:00.603779 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pv5vb\" (UniqueName: \"kubernetes.io/projected/aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e-kube-api-access-pv5vb\") pod \"thanos-querier-765ff6dd64-cql22\" (UID: \"aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e\") " pod="openshift-monitoring/thanos-querier-765ff6dd64-cql22" Jul 23 04:41:00.681979 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:00.681950 2562 generic.go:356] "Generic (PLEG): container finished" podID="a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc" containerID="dc0cc777e09d10b899f2e087c8775c272047c1d98e6824a96b81d8135a81ad49" exitCode=0 Jul 23 04:41:00.682274 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:00.682031 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6pnvh" event={"ID":"a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc","Type":"ContainerDied","Data":"dc0cc777e09d10b899f2e087c8775c272047c1d98e6824a96b81d8135a81ad49"} Jul 23 04:41:00.700722 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:00.700705 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-765ff6dd64-cql22" Jul 23 04:41:00.860225 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:00.860191 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-765ff6dd64-cql22"] Jul 23 04:41:00.976709 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:41:00.976631 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa2ca5d6_9878_4b4f_9e8d_c0288f40d18e.slice/crio-f2b837d8c89ecd9e68b748eacf3c61caebd520238f2a2ca6c39486f2de62c187 WatchSource:0}: Error finding container f2b837d8c89ecd9e68b748eacf3c61caebd520238f2a2ca6c39486f2de62c187: Status 404 returned error can't find the container with id f2b837d8c89ecd9e68b748eacf3c61caebd520238f2a2ca6c39486f2de62c187 Jul 23 04:41:01.648855 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:01.648802 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6d98b9888d-kvnzs"] Jul 23 04:41:01.664739 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:01.664713 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6d98b9888d-kvnzs" Jul 23 04:41:01.668306 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:01.668109 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 23 04:41:01.668306 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:01.668198 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 23 04:41:01.670426 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:01.669694 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 23 04:41:01.670426 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:01.669920 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-khltk\"" Jul 23 04:41:01.670426 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:01.670105 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 23 04:41:01.670426 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:01.670226 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6d98b9888d-kvnzs"] Jul 23 04:41:01.670426 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:01.670267 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-56fp7ishh5b23\"" Jul 23 04:41:01.693300 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:01.692568 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6pnvh" event={"ID":"a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc","Type":"ContainerStarted","Data":"086fb4df69a46c1ba79e4f5fbf44d17b07a2cd2de433435e0f1dac7f26e969a9"} Jul 23 04:41:01.693300 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:01.692643 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6pnvh" event={"ID":"a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc","Type":"ContainerStarted","Data":"d9534954584960716326ac7847b1597127106bcd3196f5d209759f07a5da58f8"} Jul 23 04:41:01.694864 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:01.694776 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-m5fv5" event={"ID":"3809e4cf-5052-4402-ada4-7668bdf98a74","Type":"ContainerStarted","Data":"5c04bd9d36b37ee563a14e00e7f1ea29f4fd76746e79b66d6318069901e07be9"} Jul 23 04:41:01.694967 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:01.694890 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-m5fv5" event={"ID":"3809e4cf-5052-4402-ada4-7668bdf98a74","Type":"ContainerStarted","Data":"9abdf74fa34ddf6278687c776508a416d65bd01c03c1765dce0370c1a9dfccd2"} Jul 23 04:41:01.698404 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:01.698379 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-tk424" event={"ID":"49aa1a49-bb25-47e7-9f6c-686e559571f7","Type":"ContainerStarted","Data":"bbe1e3e4079519ffdf3372c18cf1f28b41469bc01fa3d34c290f86a36b68166a"} Jul 23 04:41:01.700737 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:01.700644 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-765ff6dd64-cql22" event={"ID":"aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e","Type":"ContainerStarted","Data":"f2b837d8c89ecd9e68b748eacf3c61caebd520238f2a2ca6c39486f2de62c187"} Jul 23 04:41:01.702357 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:01.702335 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/330ee043-0fca-4da8-b619-3853fcfeb19b-metrics-server-audit-profiles\") pod \"metrics-server-6d98b9888d-kvnzs\" (UID: \"330ee043-0fca-4da8-b619-3853fcfeb19b\") " pod="openshift-monitoring/metrics-server-6d98b9888d-kvnzs" Jul 23 04:41:01.702453 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:01.702383 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhd45\" (UniqueName: \"kubernetes.io/projected/330ee043-0fca-4da8-b619-3853fcfeb19b-kube-api-access-vhd45\") pod \"metrics-server-6d98b9888d-kvnzs\" (UID: \"330ee043-0fca-4da8-b619-3853fcfeb19b\") " pod="openshift-monitoring/metrics-server-6d98b9888d-kvnzs" Jul 23 04:41:01.702453 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:01.702420 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/330ee043-0fca-4da8-b619-3853fcfeb19b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6d98b9888d-kvnzs\" (UID: \"330ee043-0fca-4da8-b619-3853fcfeb19b\") " pod="openshift-monitoring/metrics-server-6d98b9888d-kvnzs" Jul 23 04:41:01.702924 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:01.702624 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/330ee043-0fca-4da8-b619-3853fcfeb19b-client-ca-bundle\") pod \"metrics-server-6d98b9888d-kvnzs\" (UID: \"330ee043-0fca-4da8-b619-3853fcfeb19b\") " pod="openshift-monitoring/metrics-server-6d98b9888d-kvnzs" Jul 23 04:41:01.702924 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:01.702682 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/330ee043-0fca-4da8-b619-3853fcfeb19b-secret-metrics-server-tls\") pod \"metrics-server-6d98b9888d-kvnzs\" (UID: \"330ee043-0fca-4da8-b619-3853fcfeb19b\") " pod="openshift-monitoring/metrics-server-6d98b9888d-kvnzs" Jul 23 04:41:01.702924 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:01.702720 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/330ee043-0fca-4da8-b619-3853fcfeb19b-audit-log\") pod \"metrics-server-6d98b9888d-kvnzs\" (UID: \"330ee043-0fca-4da8-b619-3853fcfeb19b\") " pod="openshift-monitoring/metrics-server-6d98b9888d-kvnzs" Jul 23 04:41:01.702924 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:01.702748 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/330ee043-0fca-4da8-b619-3853fcfeb19b-secret-metrics-server-client-certs\") pod \"metrics-server-6d98b9888d-kvnzs\" (UID: \"330ee043-0fca-4da8-b619-3853fcfeb19b\") " pod="openshift-monitoring/metrics-server-6d98b9888d-kvnzs" Jul 23 04:41:01.714592 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:01.714543 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6pnvh" podStartSLOduration=2.568549013 podStartE2EDuration="4.714525008s" totalImagesPullingTime="2.021216993s" totalInitContainerRuntime="124.759002ms" isStatefulPod=true podCreationTimestamp="2026-07-23 04:40:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:41:01.71380378 +0000 UTC m=+97.056316673" watchObservedRunningTime="2026-07-23 04:41:01.714525008 +0000 UTC m=+97.057037897" Jul 23 04:41:01.737611 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:01.737549 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-tk424" podStartSLOduration=2.313385439 podStartE2EDuration="4.737529335s" totalImagesPullingTime="2.424143896s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:40:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:41:01.735210572 +0000 UTC m=+97.077723461" watchObservedRunningTime="2026-07-23 04:41:01.737529335 +0000 UTC m=+97.080042225" Jul 23 04:41:01.803800 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:01.803767 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/330ee043-0fca-4da8-b619-3853fcfeb19b-metrics-server-audit-profiles\") pod \"metrics-server-6d98b9888d-kvnzs\" (UID: \"330ee043-0fca-4da8-b619-3853fcfeb19b\") " pod="openshift-monitoring/metrics-server-6d98b9888d-kvnzs" Jul 23 04:41:01.805046 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:01.803984 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhd45\" (UniqueName: \"kubernetes.io/projected/330ee043-0fca-4da8-b619-3853fcfeb19b-kube-api-access-vhd45\") pod \"metrics-server-6d98b9888d-kvnzs\" (UID: \"330ee043-0fca-4da8-b619-3853fcfeb19b\") " pod="openshift-monitoring/metrics-server-6d98b9888d-kvnzs" Jul 23 04:41:01.805046 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:01.804079 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/330ee043-0fca-4da8-b619-3853fcfeb19b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6d98b9888d-kvnzs\" (UID: \"330ee043-0fca-4da8-b619-3853fcfeb19b\") " pod="openshift-monitoring/metrics-server-6d98b9888d-kvnzs" Jul 23 04:41:01.805046 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:01.804318 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/330ee043-0fca-4da8-b619-3853fcfeb19b-client-ca-bundle\") pod \"metrics-server-6d98b9888d-kvnzs\" (UID: \"330ee043-0fca-4da8-b619-3853fcfeb19b\") " pod="openshift-monitoring/metrics-server-6d98b9888d-kvnzs" Jul 23 04:41:01.805046 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:01.804384 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/330ee043-0fca-4da8-b619-3853fcfeb19b-secret-metrics-server-tls\") pod \"metrics-server-6d98b9888d-kvnzs\" (UID: \"330ee043-0fca-4da8-b619-3853fcfeb19b\") " pod="openshift-monitoring/metrics-server-6d98b9888d-kvnzs" Jul 23 04:41:01.805046 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:01.804412 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/330ee043-0fca-4da8-b619-3853fcfeb19b-audit-log\") pod \"metrics-server-6d98b9888d-kvnzs\" (UID: \"330ee043-0fca-4da8-b619-3853fcfeb19b\") " pod="openshift-monitoring/metrics-server-6d98b9888d-kvnzs" Jul 23 04:41:01.805046 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:01.804430 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/330ee043-0fca-4da8-b619-3853fcfeb19b-secret-metrics-server-client-certs\") pod \"metrics-server-6d98b9888d-kvnzs\" (UID: \"330ee043-0fca-4da8-b619-3853fcfeb19b\") " pod="openshift-monitoring/metrics-server-6d98b9888d-kvnzs" Jul 23 04:41:01.805046 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:01.804736 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/330ee043-0fca-4da8-b619-3853fcfeb19b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6d98b9888d-kvnzs\" (UID: \"330ee043-0fca-4da8-b619-3853fcfeb19b\") " pod="openshift-monitoring/metrics-server-6d98b9888d-kvnzs" Jul 23 04:41:01.805046 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:01.804994 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/330ee043-0fca-4da8-b619-3853fcfeb19b-metrics-server-audit-profiles\") pod \"metrics-server-6d98b9888d-kvnzs\" (UID: \"330ee043-0fca-4da8-b619-3853fcfeb19b\") " pod="openshift-monitoring/metrics-server-6d98b9888d-kvnzs" Jul 23 04:41:01.805046 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:01.805051 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/330ee043-0fca-4da8-b619-3853fcfeb19b-audit-log\") pod \"metrics-server-6d98b9888d-kvnzs\" (UID: \"330ee043-0fca-4da8-b619-3853fcfeb19b\") " pod="openshift-monitoring/metrics-server-6d98b9888d-kvnzs" Jul 23 04:41:01.807803 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:01.807778 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/330ee043-0fca-4da8-b619-3853fcfeb19b-secret-metrics-server-client-certs\") pod \"metrics-server-6d98b9888d-kvnzs\" (UID: \"330ee043-0fca-4da8-b619-3853fcfeb19b\") " pod="openshift-monitoring/metrics-server-6d98b9888d-kvnzs" Jul 23 04:41:01.808435 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:01.808396 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/330ee043-0fca-4da8-b619-3853fcfeb19b-secret-metrics-server-tls\") pod \"metrics-server-6d98b9888d-kvnzs\" (UID: \"330ee043-0fca-4da8-b619-3853fcfeb19b\") " pod="openshift-monitoring/metrics-server-6d98b9888d-kvnzs" Jul 23 04:41:01.808663 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:01.808618 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/330ee043-0fca-4da8-b619-3853fcfeb19b-client-ca-bundle\") pod \"metrics-server-6d98b9888d-kvnzs\" (UID: \"330ee043-0fca-4da8-b619-3853fcfeb19b\") " pod="openshift-monitoring/metrics-server-6d98b9888d-kvnzs" Jul 23 04:41:01.816140 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:01.816082 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhd45\" (UniqueName: \"kubernetes.io/projected/330ee043-0fca-4da8-b619-3853fcfeb19b-kube-api-access-vhd45\") pod \"metrics-server-6d98b9888d-kvnzs\" (UID: \"330ee043-0fca-4da8-b619-3853fcfeb19b\") " pod="openshift-monitoring/metrics-server-6d98b9888d-kvnzs" Jul 23 04:41:02.025208 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:02.025131 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6d98b9888d-kvnzs" Jul 23 04:41:02.194779 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:02.194751 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6d98b9888d-kvnzs"] Jul 23 04:41:02.198371 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:41:02.198340 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod330ee043_0fca_4da8_b619_3853fcfeb19b.slice/crio-46936f45b3ea09795804423d6b2c2b90582b0a69808df6dab887ac7abbac237f WatchSource:0}: Error finding container 46936f45b3ea09795804423d6b2c2b90582b0a69808df6dab887ac7abbac237f: Status 404 returned error can't find the container with id 46936f45b3ea09795804423d6b2c2b90582b0a69808df6dab887ac7abbac237f Jul 23 04:41:02.198752 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:02.198723 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c6dc857b6-2dkkn"] Jul 23 04:41:02.225194 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:02.225164 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c6dc857b6-2dkkn"] Jul 23 04:41:02.225305 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:02.225263 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c6dc857b6-2dkkn" Jul 23 04:41:02.237845 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:02.237814 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 23 04:41:02.312237 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:02.312170 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/777a4f89-ebcc-490a-bd94-6d07a9bb871c-console-oauth-config\") pod \"console-c6dc857b6-2dkkn\" (UID: \"777a4f89-ebcc-490a-bd94-6d07a9bb871c\") " pod="openshift-console/console-c6dc857b6-2dkkn" Jul 23 04:41:02.312237 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:02.312211 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/777a4f89-ebcc-490a-bd94-6d07a9bb871c-console-serving-cert\") pod \"console-c6dc857b6-2dkkn\" (UID: \"777a4f89-ebcc-490a-bd94-6d07a9bb871c\") " pod="openshift-console/console-c6dc857b6-2dkkn" Jul 23 04:41:02.312411 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:02.312243 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/777a4f89-ebcc-490a-bd94-6d07a9bb871c-oauth-serving-cert\") pod \"console-c6dc857b6-2dkkn\" (UID: \"777a4f89-ebcc-490a-bd94-6d07a9bb871c\") " pod="openshift-console/console-c6dc857b6-2dkkn" Jul 23 04:41:02.312411 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:02.312306 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98zxp\" (UniqueName: \"kubernetes.io/projected/777a4f89-ebcc-490a-bd94-6d07a9bb871c-kube-api-access-98zxp\") pod \"console-c6dc857b6-2dkkn\" (UID: \"777a4f89-ebcc-490a-bd94-6d07a9bb871c\") " pod="openshift-console/console-c6dc857b6-2dkkn" Jul 23 04:41:02.312501 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:02.312433 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/777a4f89-ebcc-490a-bd94-6d07a9bb871c-service-ca\") pod \"console-c6dc857b6-2dkkn\" (UID: \"777a4f89-ebcc-490a-bd94-6d07a9bb871c\") " pod="openshift-console/console-c6dc857b6-2dkkn" Jul 23 04:41:02.312568 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:02.312547 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/777a4f89-ebcc-490a-bd94-6d07a9bb871c-trusted-ca-bundle\") pod \"console-c6dc857b6-2dkkn\" (UID: \"777a4f89-ebcc-490a-bd94-6d07a9bb871c\") " pod="openshift-console/console-c6dc857b6-2dkkn" Jul 23 04:41:02.312645 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:02.312630 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/777a4f89-ebcc-490a-bd94-6d07a9bb871c-console-config\") pod \"console-c6dc857b6-2dkkn\" (UID: \"777a4f89-ebcc-490a-bd94-6d07a9bb871c\") " pod="openshift-console/console-c6dc857b6-2dkkn" Jul 23 04:41:02.413184 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:02.413130 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/777a4f89-ebcc-490a-bd94-6d07a9bb871c-trusted-ca-bundle\") pod \"console-c6dc857b6-2dkkn\" (UID: \"777a4f89-ebcc-490a-bd94-6d07a9bb871c\") " pod="openshift-console/console-c6dc857b6-2dkkn" Jul 23 04:41:02.413341 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:02.413203 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/777a4f89-ebcc-490a-bd94-6d07a9bb871c-console-config\") pod \"console-c6dc857b6-2dkkn\" (UID: \"777a4f89-ebcc-490a-bd94-6d07a9bb871c\") " pod="openshift-console/console-c6dc857b6-2dkkn" Jul 23 04:41:02.413341 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:02.413260 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/777a4f89-ebcc-490a-bd94-6d07a9bb871c-console-oauth-config\") pod \"console-c6dc857b6-2dkkn\" (UID: \"777a4f89-ebcc-490a-bd94-6d07a9bb871c\") " pod="openshift-console/console-c6dc857b6-2dkkn" Jul 23 04:41:02.413341 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:02.413301 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/777a4f89-ebcc-490a-bd94-6d07a9bb871c-console-serving-cert\") pod \"console-c6dc857b6-2dkkn\" (UID: \"777a4f89-ebcc-490a-bd94-6d07a9bb871c\") " pod="openshift-console/console-c6dc857b6-2dkkn" Jul 23 04:41:02.413499 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:02.413349 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/777a4f89-ebcc-490a-bd94-6d07a9bb871c-oauth-serving-cert\") pod \"console-c6dc857b6-2dkkn\" (UID: \"777a4f89-ebcc-490a-bd94-6d07a9bb871c\") " pod="openshift-console/console-c6dc857b6-2dkkn" Jul 23 04:41:02.413499 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:02.413397 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98zxp\" (UniqueName: \"kubernetes.io/projected/777a4f89-ebcc-490a-bd94-6d07a9bb871c-kube-api-access-98zxp\") pod \"console-c6dc857b6-2dkkn\" (UID: \"777a4f89-ebcc-490a-bd94-6d07a9bb871c\") " pod="openshift-console/console-c6dc857b6-2dkkn" Jul 23 04:41:02.413499 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:02.413452 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/777a4f89-ebcc-490a-bd94-6d07a9bb871c-service-ca\") pod \"console-c6dc857b6-2dkkn\" (UID: \"777a4f89-ebcc-490a-bd94-6d07a9bb871c\") " pod="openshift-console/console-c6dc857b6-2dkkn" Jul 23 04:41:02.414195 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:02.414146 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/777a4f89-ebcc-490a-bd94-6d07a9bb871c-console-config\") pod \"console-c6dc857b6-2dkkn\" (UID: \"777a4f89-ebcc-490a-bd94-6d07a9bb871c\") " pod="openshift-console/console-c6dc857b6-2dkkn" Jul 23 04:41:02.414322 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:02.414211 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/777a4f89-ebcc-490a-bd94-6d07a9bb871c-service-ca\") pod \"console-c6dc857b6-2dkkn\" (UID: \"777a4f89-ebcc-490a-bd94-6d07a9bb871c\") " pod="openshift-console/console-c6dc857b6-2dkkn" Jul 23 04:41:02.414427 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:02.414406 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/777a4f89-ebcc-490a-bd94-6d07a9bb871c-trusted-ca-bundle\") pod \"console-c6dc857b6-2dkkn\" (UID: \"777a4f89-ebcc-490a-bd94-6d07a9bb871c\") " pod="openshift-console/console-c6dc857b6-2dkkn" Jul 23 04:41:02.414484 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:02.414472 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/777a4f89-ebcc-490a-bd94-6d07a9bb871c-oauth-serving-cert\") pod \"console-c6dc857b6-2dkkn\" (UID: \"777a4f89-ebcc-490a-bd94-6d07a9bb871c\") " pod="openshift-console/console-c6dc857b6-2dkkn" Jul 23 04:41:02.417203 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:02.417156 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/777a4f89-ebcc-490a-bd94-6d07a9bb871c-console-serving-cert\") pod \"console-c6dc857b6-2dkkn\" (UID: \"777a4f89-ebcc-490a-bd94-6d07a9bb871c\") " pod="openshift-console/console-c6dc857b6-2dkkn" Jul 23 04:41:02.417705 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:02.417677 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/777a4f89-ebcc-490a-bd94-6d07a9bb871c-console-oauth-config\") pod \"console-c6dc857b6-2dkkn\" (UID: \"777a4f89-ebcc-490a-bd94-6d07a9bb871c\") " pod="openshift-console/console-c6dc857b6-2dkkn" Jul 23 04:41:02.425463 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:02.425443 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98zxp\" (UniqueName: \"kubernetes.io/projected/777a4f89-ebcc-490a-bd94-6d07a9bb871c-kube-api-access-98zxp\") pod \"console-c6dc857b6-2dkkn\" (UID: \"777a4f89-ebcc-490a-bd94-6d07a9bb871c\") " pod="openshift-console/console-c6dc857b6-2dkkn" Jul 23 04:41:02.536950 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:02.536925 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c6dc857b6-2dkkn" Jul 23 04:41:02.706526 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:02.706489 2562 generic.go:356] "Generic (PLEG): container finished" podID="bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" containerID="e94e2ddb8ad79fa91dc6d6891d4d0a7681c4c0b26fcd62a53c31183b75fc4b1c" exitCode=0 Jul 23 04:41:02.706950 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:02.706546 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd","Type":"ContainerDied","Data":"e94e2ddb8ad79fa91dc6d6891d4d0a7681c4c0b26fcd62a53c31183b75fc4b1c"} Jul 23 04:41:02.708417 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:02.708376 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6d98b9888d-kvnzs" event={"ID":"330ee043-0fca-4da8-b619-3853fcfeb19b","Type":"ContainerStarted","Data":"46936f45b3ea09795804423d6b2c2b90582b0a69808df6dab887ac7abbac237f"} Jul 23 04:41:02.711686 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:02.711658 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-m5fv5" event={"ID":"3809e4cf-5052-4402-ada4-7668bdf98a74","Type":"ContainerStarted","Data":"ba42c3ce86e5d808a1d8e5588f816e70a4be0634bb18f485a9bc5739abbd4110"} Jul 23 04:41:02.757346 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:02.757293 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-m5fv5" podStartSLOduration=3.032970751 podStartE2EDuration="5.757274075s" totalImagesPullingTime="2.724303324s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:40:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:41:02.755317614 +0000 UTC m=+98.097830502" watchObservedRunningTime="2026-07-23 04:41:02.757274075 +0000 UTC m=+98.099786964" Jul 23 04:41:03.246264 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.246240 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c6dc857b6-2dkkn"] Jul 23 04:41:03.248390 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:41:03.248360 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod777a4f89_ebcc_490a_bd94_6d07a9bb871c.slice/crio-aeef1064a92d1170de0a28892cd267137ec12c37ce00795c3c2b4cb0eb603142 WatchSource:0}: Error finding container aeef1064a92d1170de0a28892cd267137ec12c37ce00795c3c2b4cb0eb603142: Status 404 returned error can't find the container with id aeef1064a92d1170de0a28892cd267137ec12c37ce00795c3c2b4cb0eb603142 Jul 23 04:41:03.627296 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.627262 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 23 04:41:03.644387 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.644356 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.647822 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.647664 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 23 04:41:03.648252 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.648008 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 23 04:41:03.650272 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.649803 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 23 04:41:03.650272 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.650237 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 23 04:41:03.652245 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.650572 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 23 04:41:03.652245 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.650769 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 23 04:41:03.652245 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.651005 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 23 04:41:03.652245 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.651188 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 23 04:41:03.652245 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.651381 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-gvzld\"" Jul 23 04:41:03.652245 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.651862 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 23 04:41:03.652245 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.651870 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 23 04:41:03.652245 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.652053 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 23 04:41:03.652245 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.652054 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 23 04:41:03.660880 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.657241 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-edili6tdn8ege\"" Jul 23 04:41:03.660880 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.657580 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 23 04:41:03.660880 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.659570 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 23 04:41:03.663249 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.663229 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 23 04:41:03.720932 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.720895 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-765ff6dd64-cql22" event={"ID":"aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e","Type":"ContainerStarted","Data":"c890a0074ebeccaef2e4a283d802e0ccce92a100626e5c5d89575c33457f2062"} Jul 23 04:41:03.723240 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.723216 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c6dc857b6-2dkkn" event={"ID":"777a4f89-ebcc-490a-bd94-6d07a9bb871c","Type":"ContainerStarted","Data":"e0b93e0911b74e6711b1602ace57236ff040e4b91f5eb29147ef40511fec33aa"} Jul 23 04:41:03.723342 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.723255 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c6dc857b6-2dkkn" event={"ID":"777a4f89-ebcc-490a-bd94-6d07a9bb871c","Type":"ContainerStarted","Data":"aeef1064a92d1170de0a28892cd267137ec12c37ce00795c3c2b4cb0eb603142"} Jul 23 04:41:03.725777 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.725753 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.725896 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.725792 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.725896 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.725816 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.726027 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.725894 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.726027 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.725985 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wh66p\" (UniqueName: \"kubernetes.io/projected/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-kube-api-access-wh66p\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.726027 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.726022 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-config\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.726175 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.726049 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.726175 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.726084 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.726175 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.726112 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.726175 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.726140 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.726175 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.726165 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-config-out\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.726423 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.726189 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.726423 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.726238 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.726423 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.726323 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.726423 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.726372 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.726423 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.726394 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-web-config\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.726423 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.726416 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.726717 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.726455 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.726717 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.726482 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.726717 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.726508 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.827664 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.827488 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh66p\" (UniqueName: \"kubernetes.io/projected/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-kube-api-access-wh66p\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.827664 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.827564 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-config\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.827664 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.827594 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.829866 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.827629 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.829866 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.827970 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.829866 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.828003 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.829866 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.828049 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-config-out\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.829866 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.828073 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.829866 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.828102 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.829866 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.828136 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.829866 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.828161 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.829866 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.828181 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-web-config\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.829866 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.828206 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.829866 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.828262 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.829866 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.828281 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.829866 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.828309 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.829866 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.828378 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.829866 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.828407 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.829866 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.828430 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.829866 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.828458 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.830780 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.829028 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.830780 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.829063 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.830780 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.829381 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.830780 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.830002 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.830780 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.830136 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.830780 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.830428 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.830780 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.830757 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.832252 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.832141 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.832772 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.832747 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.833064 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.833044 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-config-out\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.833797 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.833773 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.835814 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.835728 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.836425 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.836328 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-config\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.836909 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.836856 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.847747 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.847701 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.848026 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.847912 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.848192 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.848170 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.848960 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.848920 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-web-config\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.849319 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.849300 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.851499 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.851458 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wh66p\" (UniqueName: \"kubernetes.io/projected/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-kube-api-access-wh66p\") pod \"prometheus-k8s-0\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:03.989528 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:03.989437 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:04.179642 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:04.179179 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c6dc857b6-2dkkn" podStartSLOduration=2.179161808 podStartE2EDuration="2.179161808s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:41:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:41:03.749806224 +0000 UTC m=+99.092319112" watchObservedRunningTime="2026-07-23 04:41:04.179161808 +0000 UTC m=+99.521674696" Jul 23 04:41:04.179642 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:04.179481 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 23 04:41:04.182687 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:41:04.182657 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3cd8820_6afb_4e8e_b2e4_6471f66d12dd.slice/crio-cb762567cccdc5cd52ecc4374109fe752717f23f7d888a3b98a56ef83998ea05 WatchSource:0}: Error finding container cb762567cccdc5cd52ecc4374109fe752717f23f7d888a3b98a56ef83998ea05: Status 404 returned error can't find the container with id cb762567cccdc5cd52ecc4374109fe752717f23f7d888a3b98a56ef83998ea05 Jul 23 04:41:04.733169 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:04.733130 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-765ff6dd64-cql22" event={"ID":"aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e","Type":"ContainerStarted","Data":"3500e93382d494c4b34410eb790422d5bd91fc91df79ad07a0d772ca400433cb"} Jul 23 04:41:04.733670 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:04.733223 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-765ff6dd64-cql22" event={"ID":"aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e","Type":"ContainerStarted","Data":"f25189e8e0dc4272c963ba5f64ecd621eca3aafd3c4db152faff56d372898c09"} Jul 23 04:41:04.735707 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:04.735676 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd","Type":"ContainerStarted","Data":"09c71d41de412cc828294002a80e3c21f19dc9ab558f9b783a20a490e85fba4e"} Jul 23 04:41:04.735870 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:04.735757 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd","Type":"ContainerStarted","Data":"cb762567cccdc5cd52ecc4374109fe752717f23f7d888a3b98a56ef83998ea05"} Jul 23 04:41:05.742742 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:05.742671 2562 generic.go:356] "Generic (PLEG): container finished" podID="f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" containerID="09c71d41de412cc828294002a80e3c21f19dc9ab558f9b783a20a490e85fba4e" exitCode=0 Jul 23 04:41:05.743197 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:05.742753 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd","Type":"ContainerDied","Data":"09c71d41de412cc828294002a80e3c21f19dc9ab558f9b783a20a490e85fba4e"} Jul 23 04:41:06.753262 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:06.753224 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-765ff6dd64-cql22" event={"ID":"aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e","Type":"ContainerStarted","Data":"7c9344b93fc873e93913da3106c6eb64e6ecc4c98e7c371ad444c75512dabd05"} Jul 23 04:41:06.753703 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:06.753362 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-765ff6dd64-cql22" event={"ID":"aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e","Type":"ContainerStarted","Data":"ac5e90004f4b068ae4ba732dae10ec326b8904906e3f7cd673635ce3b1e92cbb"} Jul 23 04:41:06.753703 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:06.753417 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-765ff6dd64-cql22" Jul 23 04:41:06.753703 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:06.753464 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-765ff6dd64-cql22" event={"ID":"aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e","Type":"ContainerStarted","Data":"85b0d8d1740aaa6738537040922eb2bb893013bfc362c931e3c544ba7b1bcd98"} Jul 23 04:41:06.757697 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:06.757632 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd","Type":"ContainerStarted","Data":"20ff28cbdfccc08dc38c6d4916352fe099fe3b92e2dee1ee5a0d79fccb2c676d"} Jul 23 04:41:06.757818 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:06.757700 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd","Type":"ContainerStarted","Data":"2a297a42758122d387b063e8cbb4020ec0c89c388534587e6708c843b44608d1"} Jul 23 04:41:06.757818 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:06.757742 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd","Type":"ContainerStarted","Data":"87bbb12dfedba7458840e2bc56be90af729cc5601817670469cd4e63b2d57c44"} Jul 23 04:41:06.757818 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:06.757780 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd","Type":"ContainerStarted","Data":"55562ac6e419366b803e4786d27d2212172bb3a566415dc17247a360b78befa1"} Jul 23 04:41:06.757818 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:06.757815 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd","Type":"ContainerStarted","Data":"96f2d3ae6dde99d643f7bba84e8faed9802b44d4b3c3a41a73446a2230da37c2"} Jul 23 04:41:06.758108 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:06.757870 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd","Type":"ContainerStarted","Data":"2ae9c063dc75538dd20a4c59ded70d09bf949a106146a12bd5a5bf3ea78cc972"} Jul 23 04:41:06.759438 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:06.759410 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6d98b9888d-kvnzs" event={"ID":"330ee043-0fca-4da8-b619-3853fcfeb19b","Type":"ContainerStarted","Data":"1d6958e9ab070053c9576778eb02861e3b6e1462a5f2ce5d2f4c2eddc4e3cda7"} Jul 23 04:41:06.791224 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:06.791173 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-765ff6dd64-cql22" podStartSLOduration=1.914532947 podStartE2EDuration="6.791155697s" totalImagesPullingTime="4.87662275s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:41:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:41:06.789419525 +0000 UTC m=+102.131932415" watchObservedRunningTime="2026-07-23 04:41:06.791155697 +0000 UTC m=+102.133668588" Jul 23 04:41:06.828034 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:06.827986 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.475254012 podStartE2EDuration="8.827971227s" totalImagesPullingTime="5.331584784s" totalInitContainerRuntime="21.132431ms" isStatefulPod=false podCreationTimestamp="2026-07-23 04:40:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:41:06.824271183 +0000 UTC m=+102.166784083" watchObservedRunningTime="2026-07-23 04:41:06.827971227 +0000 UTC m=+102.170484114" Jul 23 04:41:06.849192 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:06.848366 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6d98b9888d-kvnzs" podStartSLOduration=2.433821712 podStartE2EDuration="5.848348348s" totalImagesPullingTime="3.414526636s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:41:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:41:06.847863324 +0000 UTC m=+102.190376208" watchObservedRunningTime="2026-07-23 04:41:06.848348348 +0000 UTC m=+102.190861238" Jul 23 04:41:10.571573 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:10.571543 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-7dsv7" Jul 23 04:41:10.785913 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:10.785877 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd","Type":"ContainerStarted","Data":"6bd8ac7c228623e9bfebc9aacf0fc34033feb54058345a945c65bc97dceb8c8c"} Jul 23 04:41:10.786243 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:10.785957 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd","Type":"ContainerStarted","Data":"cb4426a1ff8c60fb8b6ad8fed02f7eafc82a8b23c5ebb6525ff8615bf3a95e3e"} Jul 23 04:41:10.786243 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:10.785999 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd","Type":"ContainerStarted","Data":"0c716638ac6e39d4f06f9f2ab725c8590c33e0d2db80f49fc9bc0c93ad6f395c"} Jul 23 04:41:10.786243 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:10.786043 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd","Type":"ContainerStarted","Data":"ca5bc1a0f6240e6068f62fa3b8f1be4b8428d222307ecb31ad1709b6e16c0b2e"} Jul 23 04:41:10.786243 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:10.786077 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd","Type":"ContainerStarted","Data":"07d023b8c66dfa8fe3f9e65bb03609c82ae8a33be9e2c72f813940b7ba065181"} Jul 23 04:41:10.786243 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:10.786113 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd","Type":"ContainerStarted","Data":"64cfd4f8db31dfc4c32932396afb36862ad3653104903254fc485711b54fa24d"} Jul 23 04:41:10.820601 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:10.820550 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.887228119 podStartE2EDuration="7.820530269s" totalImagesPullingTime="3.914746724s" totalInitContainerRuntime="18.555426ms" isStatefulPod=false podCreationTimestamp="2026-07-23 04:41:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:41:10.8177399 +0000 UTC m=+106.160252789" watchObservedRunningTime="2026-07-23 04:41:10.820530269 +0000 UTC m=+106.163043156" Jul 23 04:41:12.537376 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:12.537347 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-c6dc857b6-2dkkn" Jul 23 04:41:12.537791 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:12.537394 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c6dc857b6-2dkkn" Jul 23 04:41:12.542140 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:12.542118 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-c6dc857b6-2dkkn" Jul 23 04:41:12.773918 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:12.773892 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-765ff6dd64-cql22" Jul 23 04:41:12.805601 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:12.805537 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-c6dc857b6-2dkkn" Jul 23 04:41:13.989922 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:13.989892 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:41:22.025485 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:22.025451 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6d98b9888d-kvnzs" Jul 23 04:41:22.025985 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:22.025509 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6d98b9888d-kvnzs" Jul 23 04:41:24.814306 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:24.814246 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7675f5469-nwfsm" podUID="3c347ce4-da6a-4e19-91c9-ac29be9cef57" containerName="console" containerID="cri-o://549c82235838023178c30ffa55cc21889626b78bcbb87e1d797c0fa73bd2a97e" gracePeriod=15 Jul 23 04:41:25.104952 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:25.104928 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7675f5469-nwfsm" Jul 23 04:41:25.223008 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:25.222981 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c347ce4-da6a-4e19-91c9-ac29be9cef57-console-serving-cert\") pod \"3c347ce4-da6a-4e19-91c9-ac29be9cef57\" (UID: \"3c347ce4-da6a-4e19-91c9-ac29be9cef57\") " Jul 23 04:41:25.223161 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:25.223026 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c347ce4-da6a-4e19-91c9-ac29be9cef57-service-ca\") pod \"3c347ce4-da6a-4e19-91c9-ac29be9cef57\" (UID: \"3c347ce4-da6a-4e19-91c9-ac29be9cef57\") " Jul 23 04:41:25.223161 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:25.223066 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c347ce4-da6a-4e19-91c9-ac29be9cef57-console-oauth-config\") pod \"3c347ce4-da6a-4e19-91c9-ac29be9cef57\" (UID: \"3c347ce4-da6a-4e19-91c9-ac29be9cef57\") " Jul 23 04:41:25.223161 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:25.223120 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gjhcm\" (UniqueName: \"kubernetes.io/projected/3c347ce4-da6a-4e19-91c9-ac29be9cef57-kube-api-access-gjhcm\") pod \"3c347ce4-da6a-4e19-91c9-ac29be9cef57\" (UID: \"3c347ce4-da6a-4e19-91c9-ac29be9cef57\") " Jul 23 04:41:25.223161 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:25.223155 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c347ce4-da6a-4e19-91c9-ac29be9cef57-console-config\") pod \"3c347ce4-da6a-4e19-91c9-ac29be9cef57\" (UID: \"3c347ce4-da6a-4e19-91c9-ac29be9cef57\") " Jul 23 04:41:25.223379 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:25.223241 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c347ce4-da6a-4e19-91c9-ac29be9cef57-oauth-serving-cert\") pod \"3c347ce4-da6a-4e19-91c9-ac29be9cef57\" (UID: \"3c347ce4-da6a-4e19-91c9-ac29be9cef57\") " Jul 23 04:41:25.223539 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:25.223504 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c347ce4-da6a-4e19-91c9-ac29be9cef57-service-ca" (OuterVolumeSpecName: "service-ca") pod "3c347ce4-da6a-4e19-91c9-ac29be9cef57" (UID: "3c347ce4-da6a-4e19-91c9-ac29be9cef57"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 04:41:25.223856 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:25.223563 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c347ce4-da6a-4e19-91c9-ac29be9cef57-console-config" (OuterVolumeSpecName: "console-config") pod "3c347ce4-da6a-4e19-91c9-ac29be9cef57" (UID: "3c347ce4-da6a-4e19-91c9-ac29be9cef57"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 04:41:25.223856 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:25.223621 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c347ce4-da6a-4e19-91c9-ac29be9cef57-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3c347ce4-da6a-4e19-91c9-ac29be9cef57" (UID: "3c347ce4-da6a-4e19-91c9-ac29be9cef57"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 04:41:25.225434 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:25.225404 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c347ce4-da6a-4e19-91c9-ac29be9cef57-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3c347ce4-da6a-4e19-91c9-ac29be9cef57" (UID: "3c347ce4-da6a-4e19-91c9-ac29be9cef57"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 04:41:25.225744 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:25.225721 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c347ce4-da6a-4e19-91c9-ac29be9cef57-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3c347ce4-da6a-4e19-91c9-ac29be9cef57" (UID: "3c347ce4-da6a-4e19-91c9-ac29be9cef57"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 04:41:25.225850 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:25.225753 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c347ce4-da6a-4e19-91c9-ac29be9cef57-kube-api-access-gjhcm" (OuterVolumeSpecName: "kube-api-access-gjhcm") pod "3c347ce4-da6a-4e19-91c9-ac29be9cef57" (UID: "3c347ce4-da6a-4e19-91c9-ac29be9cef57"). InnerVolumeSpecName "kube-api-access-gjhcm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 04:41:25.324070 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:25.324041 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c347ce4-da6a-4e19-91c9-ac29be9cef57-console-config\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:41:25.324070 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:25.324066 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c347ce4-da6a-4e19-91c9-ac29be9cef57-oauth-serving-cert\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:41:25.324233 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:25.324076 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c347ce4-da6a-4e19-91c9-ac29be9cef57-console-serving-cert\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:41:25.324233 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:25.324085 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c347ce4-da6a-4e19-91c9-ac29be9cef57-service-ca\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:41:25.324233 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:25.324093 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c347ce4-da6a-4e19-91c9-ac29be9cef57-console-oauth-config\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:41:25.324233 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:25.324102 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjhcm\" (UniqueName: \"kubernetes.io/projected/3c347ce4-da6a-4e19-91c9-ac29be9cef57-kube-api-access-gjhcm\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:41:25.858277 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:25.858246 2562 generic.go:356] "Generic (PLEG): container finished" podID="3c347ce4-da6a-4e19-91c9-ac29be9cef57" containerID="549c82235838023178c30ffa55cc21889626b78bcbb87e1d797c0fa73bd2a97e" exitCode=0 Jul 23 04:41:25.858740 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:25.858301 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7675f5469-nwfsm" Jul 23 04:41:25.858740 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:25.858341 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7675f5469-nwfsm" event={"ID":"3c347ce4-da6a-4e19-91c9-ac29be9cef57","Type":"ContainerDied","Data":"549c82235838023178c30ffa55cc21889626b78bcbb87e1d797c0fa73bd2a97e"} Jul 23 04:41:25.858740 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:25.858392 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7675f5469-nwfsm" event={"ID":"3c347ce4-da6a-4e19-91c9-ac29be9cef57","Type":"ContainerDied","Data":"04aaf63018f159891d8620585fd459a10713821edce0911ba6ddcbada5511704"} Jul 23 04:41:25.858740 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:25.858415 2562 scope.go:117] "RemoveContainer" containerID="549c82235838023178c30ffa55cc21889626b78bcbb87e1d797c0fa73bd2a97e" Jul 23 04:41:25.867409 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:25.867394 2562 scope.go:117] "RemoveContainer" containerID="549c82235838023178c30ffa55cc21889626b78bcbb87e1d797c0fa73bd2a97e" Jul 23 04:41:25.867668 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:41:25.867644 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"549c82235838023178c30ffa55cc21889626b78bcbb87e1d797c0fa73bd2a97e\": container with ID starting with 549c82235838023178c30ffa55cc21889626b78bcbb87e1d797c0fa73bd2a97e not found: ID does not exist" containerID="549c82235838023178c30ffa55cc21889626b78bcbb87e1d797c0fa73bd2a97e" Jul 23 04:41:25.867737 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:25.867681 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"549c82235838023178c30ffa55cc21889626b78bcbb87e1d797c0fa73bd2a97e"} err="failed to get container status \"549c82235838023178c30ffa55cc21889626b78bcbb87e1d797c0fa73bd2a97e\": rpc error: code = NotFound desc = could not find container \"549c82235838023178c30ffa55cc21889626b78bcbb87e1d797c0fa73bd2a97e\": container with ID starting with 549c82235838023178c30ffa55cc21889626b78bcbb87e1d797c0fa73bd2a97e not found: ID does not exist" Jul 23 04:41:25.880166 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:25.880142 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7675f5469-nwfsm"] Jul 23 04:41:25.884712 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:25.884693 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7675f5469-nwfsm"] Jul 23 04:41:27.207289 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:27.207256 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c347ce4-da6a-4e19-91c9-ac29be9cef57" path="/var/lib/kubelet/pods/3c347ce4-da6a-4e19-91c9-ac29be9cef57/volumes" Jul 23 04:41:41.926589 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:41.926550 2562 generic.go:356] "Generic (PLEG): container finished" podID="aa0558ac-8329-4a03-8a88-76a2a42d49a1" containerID="1f4b3c4ac8f230a6509a51ba03c0d33825543db228471602cecafa5af61bff93" exitCode=0 Jul 23 04:41:41.927024 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:41.926623 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-8v8xp" event={"ID":"aa0558ac-8329-4a03-8a88-76a2a42d49a1","Type":"ContainerDied","Data":"1f4b3c4ac8f230a6509a51ba03c0d33825543db228471602cecafa5af61bff93"} Jul 23 04:41:41.927085 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:41.927068 2562 scope.go:117] "RemoveContainer" containerID="1f4b3c4ac8f230a6509a51ba03c0d33825543db228471602cecafa5af61bff93" Jul 23 04:41:42.031137 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:42.031112 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6d98b9888d-kvnzs" Jul 23 04:41:42.034926 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:42.034908 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6d98b9888d-kvnzs" Jul 23 04:41:42.933859 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:41:42.933784 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-8v8xp" event={"ID":"aa0558ac-8329-4a03-8a88-76a2a42d49a1","Type":"ContainerStarted","Data":"00123e890577ea44793835b01078a7ad7b58114f3ff36fe47811d89cc3a57616"} Jul 23 04:42:03.989900 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:03.989862 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:04.006342 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:04.006315 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:04.039016 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:04.038550 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:17.727472 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:17.727395 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 23 04:42:18.080633 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:18.080593 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" containerName="alertmanager" containerID="cri-o://2ae9c063dc75538dd20a4c59ded70d09bf949a106146a12bd5a5bf3ea78cc972" gracePeriod=120 Jul 23 04:42:18.080810 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:18.080671 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" containerName="kube-rbac-proxy-metric" containerID="cri-o://2a297a42758122d387b063e8cbb4020ec0c89c388534587e6708c843b44608d1" gracePeriod=120 Jul 23 04:42:18.080810 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:18.080686 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" containerName="prom-label-proxy" containerID="cri-o://20ff28cbdfccc08dc38c6d4916352fe099fe3b92e2dee1ee5a0d79fccb2c676d" gracePeriod=120 Jul 23 04:42:18.080810 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:18.080703 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" containerName="kube-rbac-proxy-web" containerID="cri-o://55562ac6e419366b803e4786d27d2212172bb3a566415dc17247a360b78befa1" gracePeriod=120 Jul 23 04:42:18.080810 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:18.080741 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" containerName="kube-rbac-proxy" containerID="cri-o://87bbb12dfedba7458840e2bc56be90af729cc5601817670469cd4e63b2d57c44" gracePeriod=120 Jul 23 04:42:18.080810 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:18.080715 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" containerName="config-reloader" containerID="cri-o://96f2d3ae6dde99d643f7bba84e8faed9802b44d4b3c3a41a73446a2230da37c2" gracePeriod=120 Jul 23 04:42:19.087458 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.087424 2562 generic.go:356] "Generic (PLEG): container finished" podID="bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" containerID="20ff28cbdfccc08dc38c6d4916352fe099fe3b92e2dee1ee5a0d79fccb2c676d" exitCode=0 Jul 23 04:42:19.087458 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.087454 2562 generic.go:356] "Generic (PLEG): container finished" podID="bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" containerID="87bbb12dfedba7458840e2bc56be90af729cc5601817670469cd4e63b2d57c44" exitCode=0 Jul 23 04:42:19.087458 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.087461 2562 generic.go:356] "Generic (PLEG): container finished" podID="bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" containerID="96f2d3ae6dde99d643f7bba84e8faed9802b44d4b3c3a41a73446a2230da37c2" exitCode=0 Jul 23 04:42:19.087904 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.087469 2562 generic.go:356] "Generic (PLEG): container finished" podID="bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" containerID="2ae9c063dc75538dd20a4c59ded70d09bf949a106146a12bd5a5bf3ea78cc972" exitCode=0 Jul 23 04:42:19.087904 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.087491 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd","Type":"ContainerDied","Data":"20ff28cbdfccc08dc38c6d4916352fe099fe3b92e2dee1ee5a0d79fccb2c676d"} Jul 23 04:42:19.087904 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.087581 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd","Type":"ContainerDied","Data":"87bbb12dfedba7458840e2bc56be90af729cc5601817670469cd4e63b2d57c44"} Jul 23 04:42:19.087904 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.087608 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd","Type":"ContainerDied","Data":"96f2d3ae6dde99d643f7bba84e8faed9802b44d4b3c3a41a73446a2230da37c2"} Jul 23 04:42:19.087904 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.087633 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd","Type":"ContainerDied","Data":"2ae9c063dc75538dd20a4c59ded70d09bf949a106146a12bd5a5bf3ea78cc972"} Jul 23 04:42:19.325077 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.325052 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:42:19.412928 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.412858 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-config-volume\") pod \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " Jul 23 04:42:19.412928 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.412900 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-secret-alertmanager-kube-rbac-proxy\") pod \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " Jul 23 04:42:19.413108 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.412941 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " Jul 23 04:42:19.413108 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.412980 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-tls-assets\") pod \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " Jul 23 04:42:19.413108 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.413008 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-web-config\") pod \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " Jul 23 04:42:19.413108 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.413040 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-secret-alertmanager-kube-rbac-proxy-web\") pod \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " Jul 23 04:42:19.413108 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.413094 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-alertmanager-trusted-ca-bundle\") pod \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " Jul 23 04:42:19.413333 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.413136 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-cluster-tls-config\") pod \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " Jul 23 04:42:19.413333 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.413173 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-alertmanager-main-db\") pod \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " Jul 23 04:42:19.413333 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.413239 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-metrics-client-ca\") pod \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " Jul 23 04:42:19.413333 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.413266 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-config-out\") pod \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " Jul 23 04:42:19.413333 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.413310 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-secret-alertmanager-main-tls\") pod \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " Jul 23 04:42:19.413572 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.413338 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-smhjq\" (UniqueName: \"kubernetes.io/projected/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-kube-api-access-smhjq\") pod \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\" (UID: \"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd\") " Jul 23 04:42:19.413572 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.413551 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" (UID: "bd0f6674-bc91-4148-8b30-7bc7ae20f2cd"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 04:42:19.413700 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.413671 2562 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:42:19.414433 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.414071 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" (UID: "bd0f6674-bc91-4148-8b30-7bc7ae20f2cd"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 04:42:19.414433 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.414364 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" (UID: "bd0f6674-bc91-4148-8b30-7bc7ae20f2cd"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 04:42:19.415763 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.415736 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" (UID: "bd0f6674-bc91-4148-8b30-7bc7ae20f2cd"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 04:42:19.416233 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.416201 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" (UID: "bd0f6674-bc91-4148-8b30-7bc7ae20f2cd"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 04:42:19.416460 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.416416 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" (UID: "bd0f6674-bc91-4148-8b30-7bc7ae20f2cd"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 04:42:19.416752 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.416727 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" (UID: "bd0f6674-bc91-4148-8b30-7bc7ae20f2cd"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 04:42:19.417493 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.417467 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-config-volume" (OuterVolumeSpecName: "config-volume") pod "bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" (UID: "bd0f6674-bc91-4148-8b30-7bc7ae20f2cd"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 04:42:19.417579 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.417487 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-config-out" (OuterVolumeSpecName: "config-out") pod "bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" (UID: "bd0f6674-bc91-4148-8b30-7bc7ae20f2cd"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 04:42:19.417656 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.417631 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-kube-api-access-smhjq" (OuterVolumeSpecName: "kube-api-access-smhjq") pod "bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" (UID: "bd0f6674-bc91-4148-8b30-7bc7ae20f2cd"). InnerVolumeSpecName "kube-api-access-smhjq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 04:42:19.418315 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.418286 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" (UID: "bd0f6674-bc91-4148-8b30-7bc7ae20f2cd"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 04:42:19.420902 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.420883 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" (UID: "bd0f6674-bc91-4148-8b30-7bc7ae20f2cd"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 04:42:19.427620 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.427597 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-web-config" (OuterVolumeSpecName: "web-config") pod "bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" (UID: "bd0f6674-bc91-4148-8b30-7bc7ae20f2cd"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 04:42:19.514414 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.514381 2562 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-metrics-client-ca\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:42:19.514414 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.514409 2562 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-config-out\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:42:19.514622 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.514423 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-secret-alertmanager-main-tls\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:42:19.514622 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.514436 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smhjq\" (UniqueName: \"kubernetes.io/projected/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-kube-api-access-smhjq\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:42:19.514622 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.514447 2562 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-config-volume\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:42:19.514622 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.514460 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:42:19.514622 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.514471 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:42:19.514622 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.514483 2562 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-tls-assets\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:42:19.514622 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.514495 2562 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-web-config\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:42:19.514622 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.514505 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:42:19.514622 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.514520 2562 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-cluster-tls-config\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:42:19.514622 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:19.514533 2562 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd-alertmanager-main-db\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:42:20.095532 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.095502 2562 generic.go:356] "Generic (PLEG): container finished" podID="bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" containerID="2a297a42758122d387b063e8cbb4020ec0c89c388534587e6708c843b44608d1" exitCode=0 Jul 23 04:42:20.095532 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.095526 2562 generic.go:356] "Generic (PLEG): container finished" podID="bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" containerID="55562ac6e419366b803e4786d27d2212172bb3a566415dc17247a360b78befa1" exitCode=0 Jul 23 04:42:20.096088 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.095565 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd","Type":"ContainerDied","Data":"2a297a42758122d387b063e8cbb4020ec0c89c388534587e6708c843b44608d1"} Jul 23 04:42:20.096088 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.095606 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 23 04:42:20.096088 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.095623 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd","Type":"ContainerDied","Data":"55562ac6e419366b803e4786d27d2212172bb3a566415dc17247a360b78befa1"} Jul 23 04:42:20.096088 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.095650 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd0f6674-bc91-4148-8b30-7bc7ae20f2cd","Type":"ContainerDied","Data":"97cb5dc3fa67cc5f257e727d72ceeeb3054d7d221494385a0746eb40c3cab740"} Jul 23 04:42:20.096088 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.095681 2562 scope.go:117] "RemoveContainer" containerID="20ff28cbdfccc08dc38c6d4916352fe099fe3b92e2dee1ee5a0d79fccb2c676d" Jul 23 04:42:20.103997 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.103979 2562 scope.go:117] "RemoveContainer" containerID="2a297a42758122d387b063e8cbb4020ec0c89c388534587e6708c843b44608d1" Jul 23 04:42:20.111100 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.111082 2562 scope.go:117] "RemoveContainer" containerID="87bbb12dfedba7458840e2bc56be90af729cc5601817670469cd4e63b2d57c44" Jul 23 04:42:20.117871 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.117854 2562 scope.go:117] "RemoveContainer" containerID="55562ac6e419366b803e4786d27d2212172bb3a566415dc17247a360b78befa1" Jul 23 04:42:20.120598 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.120572 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 23 04:42:20.125577 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.125406 2562 scope.go:117] "RemoveContainer" containerID="96f2d3ae6dde99d643f7bba84e8faed9802b44d4b3c3a41a73446a2230da37c2" Jul 23 04:42:20.129275 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.129251 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 23 04:42:20.137982 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.137958 2562 scope.go:117] "RemoveContainer" containerID="2ae9c063dc75538dd20a4c59ded70d09bf949a106146a12bd5a5bf3ea78cc972" Jul 23 04:42:20.144782 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.144768 2562 scope.go:117] "RemoveContainer" containerID="e94e2ddb8ad79fa91dc6d6891d4d0a7681c4c0b26fcd62a53c31183b75fc4b1c" Jul 23 04:42:20.151605 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.151580 2562 scope.go:117] "RemoveContainer" containerID="20ff28cbdfccc08dc38c6d4916352fe099fe3b92e2dee1ee5a0d79fccb2c676d" Jul 23 04:42:20.151948 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:42:20.151913 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20ff28cbdfccc08dc38c6d4916352fe099fe3b92e2dee1ee5a0d79fccb2c676d\": container with ID starting with 20ff28cbdfccc08dc38c6d4916352fe099fe3b92e2dee1ee5a0d79fccb2c676d not found: ID does not exist" containerID="20ff28cbdfccc08dc38c6d4916352fe099fe3b92e2dee1ee5a0d79fccb2c676d" Jul 23 04:42:20.152051 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.151957 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20ff28cbdfccc08dc38c6d4916352fe099fe3b92e2dee1ee5a0d79fccb2c676d"} err="failed to get container status \"20ff28cbdfccc08dc38c6d4916352fe099fe3b92e2dee1ee5a0d79fccb2c676d\": rpc error: code = NotFound desc = could not find container \"20ff28cbdfccc08dc38c6d4916352fe099fe3b92e2dee1ee5a0d79fccb2c676d\": container with ID starting with 20ff28cbdfccc08dc38c6d4916352fe099fe3b92e2dee1ee5a0d79fccb2c676d not found: ID does not exist" Jul 23 04:42:20.152051 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.151976 2562 scope.go:117] "RemoveContainer" containerID="2a297a42758122d387b063e8cbb4020ec0c89c388534587e6708c843b44608d1" Jul 23 04:42:20.152291 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:42:20.152272 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a297a42758122d387b063e8cbb4020ec0c89c388534587e6708c843b44608d1\": container with ID starting with 2a297a42758122d387b063e8cbb4020ec0c89c388534587e6708c843b44608d1 not found: ID does not exist" containerID="2a297a42758122d387b063e8cbb4020ec0c89c388534587e6708c843b44608d1" Jul 23 04:42:20.152357 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.152300 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a297a42758122d387b063e8cbb4020ec0c89c388534587e6708c843b44608d1"} err="failed to get container status \"2a297a42758122d387b063e8cbb4020ec0c89c388534587e6708c843b44608d1\": rpc error: code = NotFound desc = could not find container \"2a297a42758122d387b063e8cbb4020ec0c89c388534587e6708c843b44608d1\": container with ID starting with 2a297a42758122d387b063e8cbb4020ec0c89c388534587e6708c843b44608d1 not found: ID does not exist" Jul 23 04:42:20.152357 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.152325 2562 scope.go:117] "RemoveContainer" containerID="87bbb12dfedba7458840e2bc56be90af729cc5601817670469cd4e63b2d57c44" Jul 23 04:42:20.152570 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:42:20.152553 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87bbb12dfedba7458840e2bc56be90af729cc5601817670469cd4e63b2d57c44\": container with ID starting with 87bbb12dfedba7458840e2bc56be90af729cc5601817670469cd4e63b2d57c44 not found: ID does not exist" containerID="87bbb12dfedba7458840e2bc56be90af729cc5601817670469cd4e63b2d57c44" Jul 23 04:42:20.152638 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.152588 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87bbb12dfedba7458840e2bc56be90af729cc5601817670469cd4e63b2d57c44"} err="failed to get container status \"87bbb12dfedba7458840e2bc56be90af729cc5601817670469cd4e63b2d57c44\": rpc error: code = NotFound desc = could not find container \"87bbb12dfedba7458840e2bc56be90af729cc5601817670469cd4e63b2d57c44\": container with ID starting with 87bbb12dfedba7458840e2bc56be90af729cc5601817670469cd4e63b2d57c44 not found: ID does not exist" Jul 23 04:42:20.152638 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.152611 2562 scope.go:117] "RemoveContainer" containerID="55562ac6e419366b803e4786d27d2212172bb3a566415dc17247a360b78befa1" Jul 23 04:42:20.152819 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:42:20.152795 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55562ac6e419366b803e4786d27d2212172bb3a566415dc17247a360b78befa1\": container with ID starting with 55562ac6e419366b803e4786d27d2212172bb3a566415dc17247a360b78befa1 not found: ID does not exist" containerID="55562ac6e419366b803e4786d27d2212172bb3a566415dc17247a360b78befa1" Jul 23 04:42:20.152902 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.152826 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55562ac6e419366b803e4786d27d2212172bb3a566415dc17247a360b78befa1"} err="failed to get container status \"55562ac6e419366b803e4786d27d2212172bb3a566415dc17247a360b78befa1\": rpc error: code = NotFound desc = could not find container \"55562ac6e419366b803e4786d27d2212172bb3a566415dc17247a360b78befa1\": container with ID starting with 55562ac6e419366b803e4786d27d2212172bb3a566415dc17247a360b78befa1 not found: ID does not exist" Jul 23 04:42:20.152902 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.152871 2562 scope.go:117] "RemoveContainer" containerID="96f2d3ae6dde99d643f7bba84e8faed9802b44d4b3c3a41a73446a2230da37c2" Jul 23 04:42:20.153065 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:42:20.153043 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96f2d3ae6dde99d643f7bba84e8faed9802b44d4b3c3a41a73446a2230da37c2\": container with ID starting with 96f2d3ae6dde99d643f7bba84e8faed9802b44d4b3c3a41a73446a2230da37c2 not found: ID does not exist" containerID="96f2d3ae6dde99d643f7bba84e8faed9802b44d4b3c3a41a73446a2230da37c2" Jul 23 04:42:20.153118 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.153072 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96f2d3ae6dde99d643f7bba84e8faed9802b44d4b3c3a41a73446a2230da37c2"} err="failed to get container status \"96f2d3ae6dde99d643f7bba84e8faed9802b44d4b3c3a41a73446a2230da37c2\": rpc error: code = NotFound desc = could not find container \"96f2d3ae6dde99d643f7bba84e8faed9802b44d4b3c3a41a73446a2230da37c2\": container with ID starting with 96f2d3ae6dde99d643f7bba84e8faed9802b44d4b3c3a41a73446a2230da37c2 not found: ID does not exist" Jul 23 04:42:20.153118 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.153093 2562 scope.go:117] "RemoveContainer" containerID="2ae9c063dc75538dd20a4c59ded70d09bf949a106146a12bd5a5bf3ea78cc972" Jul 23 04:42:20.153274 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:42:20.153259 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ae9c063dc75538dd20a4c59ded70d09bf949a106146a12bd5a5bf3ea78cc972\": container with ID starting with 2ae9c063dc75538dd20a4c59ded70d09bf949a106146a12bd5a5bf3ea78cc972 not found: ID does not exist" containerID="2ae9c063dc75538dd20a4c59ded70d09bf949a106146a12bd5a5bf3ea78cc972" Jul 23 04:42:20.153319 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.153279 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ae9c063dc75538dd20a4c59ded70d09bf949a106146a12bd5a5bf3ea78cc972"} err="failed to get container status \"2ae9c063dc75538dd20a4c59ded70d09bf949a106146a12bd5a5bf3ea78cc972\": rpc error: code = NotFound desc = could not find container \"2ae9c063dc75538dd20a4c59ded70d09bf949a106146a12bd5a5bf3ea78cc972\": container with ID starting with 2ae9c063dc75538dd20a4c59ded70d09bf949a106146a12bd5a5bf3ea78cc972 not found: ID does not exist" Jul 23 04:42:20.153319 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.153291 2562 scope.go:117] "RemoveContainer" containerID="e94e2ddb8ad79fa91dc6d6891d4d0a7681c4c0b26fcd62a53c31183b75fc4b1c" Jul 23 04:42:20.153507 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:42:20.153491 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e94e2ddb8ad79fa91dc6d6891d4d0a7681c4c0b26fcd62a53c31183b75fc4b1c\": container with ID starting with e94e2ddb8ad79fa91dc6d6891d4d0a7681c4c0b26fcd62a53c31183b75fc4b1c not found: ID does not exist" containerID="e94e2ddb8ad79fa91dc6d6891d4d0a7681c4c0b26fcd62a53c31183b75fc4b1c" Jul 23 04:42:20.153545 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.153512 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e94e2ddb8ad79fa91dc6d6891d4d0a7681c4c0b26fcd62a53c31183b75fc4b1c"} err="failed to get container status \"e94e2ddb8ad79fa91dc6d6891d4d0a7681c4c0b26fcd62a53c31183b75fc4b1c\": rpc error: code = NotFound desc = could not find container \"e94e2ddb8ad79fa91dc6d6891d4d0a7681c4c0b26fcd62a53c31183b75fc4b1c\": container with ID starting with e94e2ddb8ad79fa91dc6d6891d4d0a7681c4c0b26fcd62a53c31183b75fc4b1c not found: ID does not exist" Jul 23 04:42:20.153545 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.153527 2562 scope.go:117] "RemoveContainer" containerID="20ff28cbdfccc08dc38c6d4916352fe099fe3b92e2dee1ee5a0d79fccb2c676d" Jul 23 04:42:20.153751 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.153734 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20ff28cbdfccc08dc38c6d4916352fe099fe3b92e2dee1ee5a0d79fccb2c676d"} err="failed to get container status \"20ff28cbdfccc08dc38c6d4916352fe099fe3b92e2dee1ee5a0d79fccb2c676d\": rpc error: code = NotFound desc = could not find container \"20ff28cbdfccc08dc38c6d4916352fe099fe3b92e2dee1ee5a0d79fccb2c676d\": container with ID starting with 20ff28cbdfccc08dc38c6d4916352fe099fe3b92e2dee1ee5a0d79fccb2c676d not found: ID does not exist" Jul 23 04:42:20.153821 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.153752 2562 scope.go:117] "RemoveContainer" containerID="2a297a42758122d387b063e8cbb4020ec0c89c388534587e6708c843b44608d1" Jul 23 04:42:20.153983 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.153964 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a297a42758122d387b063e8cbb4020ec0c89c388534587e6708c843b44608d1"} err="failed to get container status \"2a297a42758122d387b063e8cbb4020ec0c89c388534587e6708c843b44608d1\": rpc error: code = NotFound desc = could not find container \"2a297a42758122d387b063e8cbb4020ec0c89c388534587e6708c843b44608d1\": container with ID starting with 2a297a42758122d387b063e8cbb4020ec0c89c388534587e6708c843b44608d1 not found: ID does not exist" Jul 23 04:42:20.154032 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.153983 2562 scope.go:117] "RemoveContainer" containerID="87bbb12dfedba7458840e2bc56be90af729cc5601817670469cd4e63b2d57c44" Jul 23 04:42:20.154192 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.154172 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87bbb12dfedba7458840e2bc56be90af729cc5601817670469cd4e63b2d57c44"} err="failed to get container status \"87bbb12dfedba7458840e2bc56be90af729cc5601817670469cd4e63b2d57c44\": rpc error: code = NotFound desc = could not find container \"87bbb12dfedba7458840e2bc56be90af729cc5601817670469cd4e63b2d57c44\": container with ID starting with 87bbb12dfedba7458840e2bc56be90af729cc5601817670469cd4e63b2d57c44 not found: ID does not exist" Jul 23 04:42:20.154235 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.154194 2562 scope.go:117] "RemoveContainer" containerID="55562ac6e419366b803e4786d27d2212172bb3a566415dc17247a360b78befa1" Jul 23 04:42:20.154402 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.154385 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55562ac6e419366b803e4786d27d2212172bb3a566415dc17247a360b78befa1"} err="failed to get container status \"55562ac6e419366b803e4786d27d2212172bb3a566415dc17247a360b78befa1\": rpc error: code = NotFound desc = could not find container \"55562ac6e419366b803e4786d27d2212172bb3a566415dc17247a360b78befa1\": container with ID starting with 55562ac6e419366b803e4786d27d2212172bb3a566415dc17247a360b78befa1 not found: ID does not exist" Jul 23 04:42:20.154470 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.154404 2562 scope.go:117] "RemoveContainer" containerID="96f2d3ae6dde99d643f7bba84e8faed9802b44d4b3c3a41a73446a2230da37c2" Jul 23 04:42:20.154633 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.154616 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96f2d3ae6dde99d643f7bba84e8faed9802b44d4b3c3a41a73446a2230da37c2"} err="failed to get container status \"96f2d3ae6dde99d643f7bba84e8faed9802b44d4b3c3a41a73446a2230da37c2\": rpc error: code = NotFound desc = could not find container \"96f2d3ae6dde99d643f7bba84e8faed9802b44d4b3c3a41a73446a2230da37c2\": container with ID starting with 96f2d3ae6dde99d643f7bba84e8faed9802b44d4b3c3a41a73446a2230da37c2 not found: ID does not exist" Jul 23 04:42:20.154677 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.154633 2562 scope.go:117] "RemoveContainer" containerID="2ae9c063dc75538dd20a4c59ded70d09bf949a106146a12bd5a5bf3ea78cc972" Jul 23 04:42:20.154820 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.154803 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ae9c063dc75538dd20a4c59ded70d09bf949a106146a12bd5a5bf3ea78cc972"} err="failed to get container status \"2ae9c063dc75538dd20a4c59ded70d09bf949a106146a12bd5a5bf3ea78cc972\": rpc error: code = NotFound desc = could not find container \"2ae9c063dc75538dd20a4c59ded70d09bf949a106146a12bd5a5bf3ea78cc972\": container with ID starting with 2ae9c063dc75538dd20a4c59ded70d09bf949a106146a12bd5a5bf3ea78cc972 not found: ID does not exist" Jul 23 04:42:20.154879 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.154821 2562 scope.go:117] "RemoveContainer" containerID="e94e2ddb8ad79fa91dc6d6891d4d0a7681c4c0b26fcd62a53c31183b75fc4b1c" Jul 23 04:42:20.155074 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:20.155057 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e94e2ddb8ad79fa91dc6d6891d4d0a7681c4c0b26fcd62a53c31183b75fc4b1c"} err="failed to get container status \"e94e2ddb8ad79fa91dc6d6891d4d0a7681c4c0b26fcd62a53c31183b75fc4b1c\": rpc error: code = NotFound desc = could not find container \"e94e2ddb8ad79fa91dc6d6891d4d0a7681c4c0b26fcd62a53c31183b75fc4b1c\": container with ID starting with e94e2ddb8ad79fa91dc6d6891d4d0a7681c4c0b26fcd62a53c31183b75fc4b1c not found: ID does not exist" Jul 23 04:42:21.207093 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:21.207060 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" path="/var/lib/kubelet/pods/bd0f6674-bc91-4148-8b30-7bc7ae20f2cd/volumes" Jul 23 04:42:22.047334 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.047292 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 23 04:42:22.047630 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.047587 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" containerName="prometheus" containerID="cri-o://6bd8ac7c228623e9bfebc9aacf0fc34033feb54058345a945c65bc97dceb8c8c" gracePeriod=600 Jul 23 04:42:22.047630 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.047609 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" containerName="kube-rbac-proxy" containerID="cri-o://0c716638ac6e39d4f06f9f2ab725c8590c33e0d2db80f49fc9bc0c93ad6f395c" gracePeriod=600 Jul 23 04:42:22.047630 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.047628 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" containerName="thanos-sidecar" containerID="cri-o://07d023b8c66dfa8fe3f9e65bb03609c82ae8a33be9e2c72f813940b7ba065181" gracePeriod=600 Jul 23 04:42:22.047948 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.047663 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" containerName="config-reloader" containerID="cri-o://64cfd4f8db31dfc4c32932396afb36862ad3653104903254fc485711b54fa24d" gracePeriod=600 Jul 23 04:42:22.047948 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.047709 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" containerName="kube-rbac-proxy-web" containerID="cri-o://ca5bc1a0f6240e6068f62fa3b8f1be4b8428d222307ecb31ad1709b6e16c0b2e" gracePeriod=600 Jul 23 04:42:22.047948 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.047746 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" containerName="kube-rbac-proxy-thanos" containerID="cri-o://cb4426a1ff8c60fb8b6ad8fed02f7eafc82a8b23c5ebb6525ff8615bf3a95e3e" gracePeriod=600 Jul 23 04:42:22.315745 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.315715 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:22.443986 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.443946 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-secret-grpc-tls\") pod \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " Jul 23 04:42:22.444171 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.443992 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-secret-metrics-client-certs\") pod \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " Jul 23 04:42:22.444171 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.444035 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-configmap-kubelet-serving-ca-bundle\") pod \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " Jul 23 04:42:22.444171 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.444066 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-prometheus-k8s-rulefiles-2\") pod \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " Jul 23 04:42:22.444171 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.444093 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-prometheus-k8s-rulefiles-1\") pod \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " Jul 23 04:42:22.444171 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.444122 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-config-out\") pod \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " Jul 23 04:42:22.444171 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.444145 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-prometheus-k8s-db\") pod \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " Jul 23 04:42:22.444171 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.444170 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-web-config\") pod \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " Jul 23 04:42:22.444521 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.444198 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " Jul 23 04:42:22.444521 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.444229 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-configmap-serving-certs-ca-bundle\") pod \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " Jul 23 04:42:22.444521 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.444258 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-config\") pod \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " Jul 23 04:42:22.444521 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.444317 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-configmap-metrics-client-ca\") pod \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " Jul 23 04:42:22.444521 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.444347 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-prometheus-trusted-ca-bundle\") pod \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " Jul 23 04:42:22.444521 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.444384 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wh66p\" (UniqueName: \"kubernetes.io/projected/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-kube-api-access-wh66p\") pod \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " Jul 23 04:42:22.444521 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.444409 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-prometheus-k8s-rulefiles-0\") pod \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " Jul 23 04:42:22.444521 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.444439 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-secret-kube-rbac-proxy\") pod \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " Jul 23 04:42:22.444521 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.444434 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" (UID: "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 04:42:22.444521 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.444462 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" (UID: "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 04:42:22.444521 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.444478 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " Jul 23 04:42:22.444521 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.444516 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" (UID: "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 04:42:22.445120 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.444530 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-thanos-prometheus-http-client-file\") pod \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " Jul 23 04:42:22.445120 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.444565 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-tls-assets\") pod \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " Jul 23 04:42:22.445120 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.444624 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-secret-prometheus-k8s-tls\") pod \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\" (UID: \"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd\") " Jul 23 04:42:22.445120 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.444919 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:42:22.445120 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.444939 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:42:22.445120 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.444954 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:42:22.445417 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.445396 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" (UID: "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 04:42:22.446260 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.445756 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" (UID: "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 04:42:22.446260 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.446011 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" (UID: "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 04:42:22.446485 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.446410 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" (UID: "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 04:42:22.447147 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.447065 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-config" (OuterVolumeSpecName: "config") pod "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" (UID: "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 04:42:22.447260 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.447155 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" (UID: "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 04:42:22.447593 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.447462 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" (UID: "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 04:42:22.447593 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.447565 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" (UID: "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 04:42:22.447877 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.447812 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" (UID: "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 04:42:22.448398 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.448343 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-config-out" (OuterVolumeSpecName: "config-out") pod "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" (UID: "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 04:42:22.448985 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.448963 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" (UID: "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 04:42:22.449062 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.448988 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-kube-api-access-wh66p" (OuterVolumeSpecName: "kube-api-access-wh66p") pod "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" (UID: "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd"). InnerVolumeSpecName "kube-api-access-wh66p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 04:42:22.449479 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.449456 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" (UID: "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 04:42:22.449535 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.449473 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" (UID: "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 04:42:22.449535 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.449493 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" (UID: "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 04:42:22.449650 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.449635 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" (UID: "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 04:42:22.459211 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.459189 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-web-config" (OuterVolumeSpecName: "web-config") pod "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" (UID: "f3cd8820-6afb-4e8e-b2e4-6471f66d12dd"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 04:42:22.545311 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.545278 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-configmap-metrics-client-ca\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:42:22.545311 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.545304 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-prometheus-trusted-ca-bundle\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:42:22.545311 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.545315 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wh66p\" (UniqueName: \"kubernetes.io/projected/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-kube-api-access-wh66p\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:42:22.545530 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.545324 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:42:22.545530 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.545333 2562 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-secret-kube-rbac-proxy\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:42:22.545530 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.545342 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:42:22.545530 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.545351 2562 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-thanos-prometheus-http-client-file\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:42:22.545530 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.545360 2562 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-tls-assets\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:42:22.545530 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.545370 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-secret-prometheus-k8s-tls\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:42:22.545530 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.545377 2562 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-secret-grpc-tls\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:42:22.545530 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.545385 2562 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-secret-metrics-client-certs\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:42:22.545530 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.545394 2562 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-config-out\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:42:22.545530 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.545401 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-prometheus-k8s-db\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:42:22.545530 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.545409 2562 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-web-config\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:42:22.545530 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.545418 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:42:22.545530 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.545425 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:42:22.545530 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:22.545433 2562 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd-config\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:42:23.119652 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.119620 2562 generic.go:356] "Generic (PLEG): container finished" podID="f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" containerID="cb4426a1ff8c60fb8b6ad8fed02f7eafc82a8b23c5ebb6525ff8615bf3a95e3e" exitCode=0 Jul 23 04:42:23.119652 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.119648 2562 generic.go:356] "Generic (PLEG): container finished" podID="f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" containerID="0c716638ac6e39d4f06f9f2ab725c8590c33e0d2db80f49fc9bc0c93ad6f395c" exitCode=0 Jul 23 04:42:23.119652 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.119657 2562 generic.go:356] "Generic (PLEG): container finished" podID="f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" containerID="ca5bc1a0f6240e6068f62fa3b8f1be4b8428d222307ecb31ad1709b6e16c0b2e" exitCode=0 Jul 23 04:42:23.119978 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.119665 2562 generic.go:356] "Generic (PLEG): container finished" podID="f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" containerID="07d023b8c66dfa8fe3f9e65bb03609c82ae8a33be9e2c72f813940b7ba065181" exitCode=0 Jul 23 04:42:23.119978 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.119672 2562 generic.go:356] "Generic (PLEG): container finished" podID="f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" containerID="64cfd4f8db31dfc4c32932396afb36862ad3653104903254fc485711b54fa24d" exitCode=0 Jul 23 04:42:23.119978 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.119679 2562 generic.go:356] "Generic (PLEG): container finished" podID="f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" containerID="6bd8ac7c228623e9bfebc9aacf0fc34033feb54058345a945c65bc97dceb8c8c" exitCode=0 Jul 23 04:42:23.119978 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.119693 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.119978 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.119707 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd","Type":"ContainerDied","Data":"cb4426a1ff8c60fb8b6ad8fed02f7eafc82a8b23c5ebb6525ff8615bf3a95e3e"} Jul 23 04:42:23.119978 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.119778 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd","Type":"ContainerDied","Data":"0c716638ac6e39d4f06f9f2ab725c8590c33e0d2db80f49fc9bc0c93ad6f395c"} Jul 23 04:42:23.119978 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.119807 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd","Type":"ContainerDied","Data":"ca5bc1a0f6240e6068f62fa3b8f1be4b8428d222307ecb31ad1709b6e16c0b2e"} Jul 23 04:42:23.119978 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.119860 2562 scope.go:117] "RemoveContainer" containerID="cb4426a1ff8c60fb8b6ad8fed02f7eafc82a8b23c5ebb6525ff8615bf3a95e3e" Jul 23 04:42:23.119978 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.119861 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd","Type":"ContainerDied","Data":"07d023b8c66dfa8fe3f9e65bb03609c82ae8a33be9e2c72f813940b7ba065181"} Jul 23 04:42:23.119978 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.119893 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd","Type":"ContainerDied","Data":"64cfd4f8db31dfc4c32932396afb36862ad3653104903254fc485711b54fa24d"} Jul 23 04:42:23.119978 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.119924 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd","Type":"ContainerDied","Data":"6bd8ac7c228623e9bfebc9aacf0fc34033feb54058345a945c65bc97dceb8c8c"} Jul 23 04:42:23.119978 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.119950 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3cd8820-6afb-4e8e-b2e4-6471f66d12dd","Type":"ContainerDied","Data":"cb762567cccdc5cd52ecc4374109fe752717f23f7d888a3b98a56ef83998ea05"} Jul 23 04:42:23.128047 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.128031 2562 scope.go:117] "RemoveContainer" containerID="0c716638ac6e39d4f06f9f2ab725c8590c33e0d2db80f49fc9bc0c93ad6f395c" Jul 23 04:42:23.135136 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.135114 2562 scope.go:117] "RemoveContainer" containerID="ca5bc1a0f6240e6068f62fa3b8f1be4b8428d222307ecb31ad1709b6e16c0b2e" Jul 23 04:42:23.142616 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.142599 2562 scope.go:117] "RemoveContainer" containerID="07d023b8c66dfa8fe3f9e65bb03609c82ae8a33be9e2c72f813940b7ba065181" Jul 23 04:42:23.144626 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.144606 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 23 04:42:23.149856 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.149812 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 23 04:42:23.152388 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.152370 2562 scope.go:117] "RemoveContainer" containerID="64cfd4f8db31dfc4c32932396afb36862ad3653104903254fc485711b54fa24d" Jul 23 04:42:23.162982 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.162935 2562 scope.go:117] "RemoveContainer" containerID="6bd8ac7c228623e9bfebc9aacf0fc34033feb54058345a945c65bc97dceb8c8c" Jul 23 04:42:23.176919 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.176893 2562 scope.go:117] "RemoveContainer" containerID="09c71d41de412cc828294002a80e3c21f19dc9ab558f9b783a20a490e85fba4e" Jul 23 04:42:23.178974 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.178951 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 23 04:42:23.179262 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179249 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" containerName="config-reloader" Jul 23 04:42:23.179262 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179262 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" containerName="config-reloader" Jul 23 04:42:23.179382 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179270 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" containerName="kube-rbac-proxy-thanos" Jul 23 04:42:23.179382 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179276 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" containerName="kube-rbac-proxy-thanos" Jul 23 04:42:23.179382 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179285 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" containerName="kube-rbac-proxy-web" Jul 23 04:42:23.179382 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179290 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" containerName="kube-rbac-proxy-web" Jul 23 04:42:23.179382 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179300 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" containerName="thanos-sidecar" Jul 23 04:42:23.179382 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179305 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" containerName="thanos-sidecar" Jul 23 04:42:23.179382 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179310 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" containerName="kube-rbac-proxy" Jul 23 04:42:23.179382 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179315 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" containerName="kube-rbac-proxy" Jul 23 04:42:23.179382 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179322 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" containerName="config-reloader" Jul 23 04:42:23.179382 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179327 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" containerName="config-reloader" Jul 23 04:42:23.179382 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179335 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" containerName="init-config-reloader" Jul 23 04:42:23.179382 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179340 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" containerName="init-config-reloader" Jul 23 04:42:23.179382 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179347 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" containerName="kube-rbac-proxy" Jul 23 04:42:23.179382 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179352 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" containerName="kube-rbac-proxy" Jul 23 04:42:23.179382 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179357 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" containerName="prom-label-proxy" Jul 23 04:42:23.179382 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179362 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" containerName="prom-label-proxy" Jul 23 04:42:23.179382 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179367 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" containerName="prometheus" Jul 23 04:42:23.179382 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179372 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" containerName="prometheus" Jul 23 04:42:23.179382 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179379 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" containerName="kube-rbac-proxy-metric" Jul 23 04:42:23.179382 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179384 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" containerName="kube-rbac-proxy-metric" Jul 23 04:42:23.179382 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179390 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" containerName="kube-rbac-proxy-web" Jul 23 04:42:23.179382 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179395 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" containerName="kube-rbac-proxy-web" Jul 23 04:42:23.180536 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179404 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" containerName="init-config-reloader" Jul 23 04:42:23.180536 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179408 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" containerName="init-config-reloader" Jul 23 04:42:23.180536 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179414 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" containerName="alertmanager" Jul 23 04:42:23.180536 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179419 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" containerName="alertmanager" Jul 23 04:42:23.180536 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179426 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c347ce4-da6a-4e19-91c9-ac29be9cef57" containerName="console" Jul 23 04:42:23.180536 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179431 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c347ce4-da6a-4e19-91c9-ac29be9cef57" containerName="console" Jul 23 04:42:23.180536 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179476 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" containerName="kube-rbac-proxy-web" Jul 23 04:42:23.180536 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179482 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" containerName="thanos-sidecar" Jul 23 04:42:23.180536 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179489 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" containerName="prometheus" Jul 23 04:42:23.180536 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179495 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" containerName="config-reloader" Jul 23 04:42:23.180536 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179501 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" containerName="kube-rbac-proxy" Jul 23 04:42:23.180536 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179506 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" containerName="kube-rbac-proxy-web" Jul 23 04:42:23.180536 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179511 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" containerName="kube-rbac-proxy-thanos" Jul 23 04:42:23.180536 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179517 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c347ce4-da6a-4e19-91c9-ac29be9cef57" containerName="console" Jul 23 04:42:23.180536 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179522 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" containerName="kube-rbac-proxy" Jul 23 04:42:23.180536 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179528 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" containerName="alertmanager" Jul 23 04:42:23.180536 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179533 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" containerName="config-reloader" Jul 23 04:42:23.180536 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179538 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" containerName="kube-rbac-proxy-metric" Jul 23 04:42:23.180536 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.179543 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd0f6674-bc91-4148-8b30-7bc7ae20f2cd" containerName="prom-label-proxy" Jul 23 04:42:23.183358 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.183337 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.185388 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.185366 2562 scope.go:117] "RemoveContainer" containerID="cb4426a1ff8c60fb8b6ad8fed02f7eafc82a8b23c5ebb6525ff8615bf3a95e3e" Jul 23 04:42:23.185659 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:42:23.185643 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb4426a1ff8c60fb8b6ad8fed02f7eafc82a8b23c5ebb6525ff8615bf3a95e3e\": container with ID starting with cb4426a1ff8c60fb8b6ad8fed02f7eafc82a8b23c5ebb6525ff8615bf3a95e3e not found: ID does not exist" containerID="cb4426a1ff8c60fb8b6ad8fed02f7eafc82a8b23c5ebb6525ff8615bf3a95e3e" Jul 23 04:42:23.185711 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.185666 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb4426a1ff8c60fb8b6ad8fed02f7eafc82a8b23c5ebb6525ff8615bf3a95e3e"} err="failed to get container status \"cb4426a1ff8c60fb8b6ad8fed02f7eafc82a8b23c5ebb6525ff8615bf3a95e3e\": rpc error: code = NotFound desc = could not find container \"cb4426a1ff8c60fb8b6ad8fed02f7eafc82a8b23c5ebb6525ff8615bf3a95e3e\": container with ID starting with cb4426a1ff8c60fb8b6ad8fed02f7eafc82a8b23c5ebb6525ff8615bf3a95e3e not found: ID does not exist" Jul 23 04:42:23.185711 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.185681 2562 scope.go:117] "RemoveContainer" containerID="0c716638ac6e39d4f06f9f2ab725c8590c33e0d2db80f49fc9bc0c93ad6f395c" Jul 23 04:42:23.185973 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:42:23.185948 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c716638ac6e39d4f06f9f2ab725c8590c33e0d2db80f49fc9bc0c93ad6f395c\": container with ID starting with 0c716638ac6e39d4f06f9f2ab725c8590c33e0d2db80f49fc9bc0c93ad6f395c not found: ID does not exist" containerID="0c716638ac6e39d4f06f9f2ab725c8590c33e0d2db80f49fc9bc0c93ad6f395c" Jul 23 04:42:23.186050 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.185979 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c716638ac6e39d4f06f9f2ab725c8590c33e0d2db80f49fc9bc0c93ad6f395c"} err="failed to get container status \"0c716638ac6e39d4f06f9f2ab725c8590c33e0d2db80f49fc9bc0c93ad6f395c\": rpc error: code = NotFound desc = could not find container \"0c716638ac6e39d4f06f9f2ab725c8590c33e0d2db80f49fc9bc0c93ad6f395c\": container with ID starting with 0c716638ac6e39d4f06f9f2ab725c8590c33e0d2db80f49fc9bc0c93ad6f395c not found: ID does not exist" Jul 23 04:42:23.186050 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.186001 2562 scope.go:117] "RemoveContainer" containerID="ca5bc1a0f6240e6068f62fa3b8f1be4b8428d222307ecb31ad1709b6e16c0b2e" Jul 23 04:42:23.186183 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.186169 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 23 04:42:23.186609 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:42:23.186254 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca5bc1a0f6240e6068f62fa3b8f1be4b8428d222307ecb31ad1709b6e16c0b2e\": container with ID starting with ca5bc1a0f6240e6068f62fa3b8f1be4b8428d222307ecb31ad1709b6e16c0b2e not found: ID does not exist" containerID="ca5bc1a0f6240e6068f62fa3b8f1be4b8428d222307ecb31ad1709b6e16c0b2e" Jul 23 04:42:23.186609 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.186286 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca5bc1a0f6240e6068f62fa3b8f1be4b8428d222307ecb31ad1709b6e16c0b2e"} err="failed to get container status \"ca5bc1a0f6240e6068f62fa3b8f1be4b8428d222307ecb31ad1709b6e16c0b2e\": rpc error: code = NotFound desc = could not find container \"ca5bc1a0f6240e6068f62fa3b8f1be4b8428d222307ecb31ad1709b6e16c0b2e\": container with ID starting with ca5bc1a0f6240e6068f62fa3b8f1be4b8428d222307ecb31ad1709b6e16c0b2e not found: ID does not exist" Jul 23 04:42:23.186609 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.186263 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 23 04:42:23.186609 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.186309 2562 scope.go:117] "RemoveContainer" containerID="07d023b8c66dfa8fe3f9e65bb03609c82ae8a33be9e2c72f813940b7ba065181" Jul 23 04:42:23.186609 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.186301 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 23 04:42:23.186609 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.186352 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 23 04:42:23.186609 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.186408 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-edili6tdn8ege\"" Jul 23 04:42:23.186609 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.186420 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 23 04:42:23.186609 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.186430 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 23 04:42:23.187082 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.186920 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 23 04:42:23.187138 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.187107 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-gvzld\"" Jul 23 04:42:23.188077 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.187287 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 23 04:42:23.188077 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.187354 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 23 04:42:23.188077 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.187482 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 23 04:42:23.188077 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.187303 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 23 04:42:23.188077 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.187846 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 23 04:42:23.189612 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:42:23.189171 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07d023b8c66dfa8fe3f9e65bb03609c82ae8a33be9e2c72f813940b7ba065181\": container with ID starting with 07d023b8c66dfa8fe3f9e65bb03609c82ae8a33be9e2c72f813940b7ba065181 not found: ID does not exist" containerID="07d023b8c66dfa8fe3f9e65bb03609c82ae8a33be9e2c72f813940b7ba065181" Jul 23 04:42:23.189612 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.189207 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07d023b8c66dfa8fe3f9e65bb03609c82ae8a33be9e2c72f813940b7ba065181"} err="failed to get container status \"07d023b8c66dfa8fe3f9e65bb03609c82ae8a33be9e2c72f813940b7ba065181\": rpc error: code = NotFound desc = could not find container \"07d023b8c66dfa8fe3f9e65bb03609c82ae8a33be9e2c72f813940b7ba065181\": container with ID starting with 07d023b8c66dfa8fe3f9e65bb03609c82ae8a33be9e2c72f813940b7ba065181 not found: ID does not exist" Jul 23 04:42:23.189612 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.189229 2562 scope.go:117] "RemoveContainer" containerID="64cfd4f8db31dfc4c32932396afb36862ad3653104903254fc485711b54fa24d" Jul 23 04:42:23.189967 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:42:23.189943 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64cfd4f8db31dfc4c32932396afb36862ad3653104903254fc485711b54fa24d\": container with ID starting with 64cfd4f8db31dfc4c32932396afb36862ad3653104903254fc485711b54fa24d not found: ID does not exist" containerID="64cfd4f8db31dfc4c32932396afb36862ad3653104903254fc485711b54fa24d" Jul 23 04:42:23.190101 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.190071 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64cfd4f8db31dfc4c32932396afb36862ad3653104903254fc485711b54fa24d"} err="failed to get container status \"64cfd4f8db31dfc4c32932396afb36862ad3653104903254fc485711b54fa24d\": rpc error: code = NotFound desc = could not find container \"64cfd4f8db31dfc4c32932396afb36862ad3653104903254fc485711b54fa24d\": container with ID starting with 64cfd4f8db31dfc4c32932396afb36862ad3653104903254fc485711b54fa24d not found: ID does not exist" Jul 23 04:42:23.190228 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.190105 2562 scope.go:117] "RemoveContainer" containerID="6bd8ac7c228623e9bfebc9aacf0fc34033feb54058345a945c65bc97dceb8c8c" Jul 23 04:42:23.194434 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:42:23.190402 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bd8ac7c228623e9bfebc9aacf0fc34033feb54058345a945c65bc97dceb8c8c\": container with ID starting with 6bd8ac7c228623e9bfebc9aacf0fc34033feb54058345a945c65bc97dceb8c8c not found: ID does not exist" containerID="6bd8ac7c228623e9bfebc9aacf0fc34033feb54058345a945c65bc97dceb8c8c" Jul 23 04:42:23.194434 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.190825 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bd8ac7c228623e9bfebc9aacf0fc34033feb54058345a945c65bc97dceb8c8c"} err="failed to get container status \"6bd8ac7c228623e9bfebc9aacf0fc34033feb54058345a945c65bc97dceb8c8c\": rpc error: code = NotFound desc = could not find container \"6bd8ac7c228623e9bfebc9aacf0fc34033feb54058345a945c65bc97dceb8c8c\": container with ID starting with 6bd8ac7c228623e9bfebc9aacf0fc34033feb54058345a945c65bc97dceb8c8c not found: ID does not exist" Jul 23 04:42:23.194434 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.190880 2562 scope.go:117] "RemoveContainer" containerID="09c71d41de412cc828294002a80e3c21f19dc9ab558f9b783a20a490e85fba4e" Jul 23 04:42:23.194434 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.191262 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 23 04:42:23.195210 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:42:23.195179 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09c71d41de412cc828294002a80e3c21f19dc9ab558f9b783a20a490e85fba4e\": container with ID starting with 09c71d41de412cc828294002a80e3c21f19dc9ab558f9b783a20a490e85fba4e not found: ID does not exist" containerID="09c71d41de412cc828294002a80e3c21f19dc9ab558f9b783a20a490e85fba4e" Jul 23 04:42:23.195307 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.195212 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09c71d41de412cc828294002a80e3c21f19dc9ab558f9b783a20a490e85fba4e"} err="failed to get container status \"09c71d41de412cc828294002a80e3c21f19dc9ab558f9b783a20a490e85fba4e\": rpc error: code = NotFound desc = could not find container \"09c71d41de412cc828294002a80e3c21f19dc9ab558f9b783a20a490e85fba4e\": container with ID starting with 09c71d41de412cc828294002a80e3c21f19dc9ab558f9b783a20a490e85fba4e not found: ID does not exist" Jul 23 04:42:23.195307 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.195241 2562 scope.go:117] "RemoveContainer" containerID="cb4426a1ff8c60fb8b6ad8fed02f7eafc82a8b23c5ebb6525ff8615bf3a95e3e" Jul 23 04:42:23.195728 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.195702 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb4426a1ff8c60fb8b6ad8fed02f7eafc82a8b23c5ebb6525ff8615bf3a95e3e"} err="failed to get container status \"cb4426a1ff8c60fb8b6ad8fed02f7eafc82a8b23c5ebb6525ff8615bf3a95e3e\": rpc error: code = NotFound desc = could not find container \"cb4426a1ff8c60fb8b6ad8fed02f7eafc82a8b23c5ebb6525ff8615bf3a95e3e\": container with ID starting with cb4426a1ff8c60fb8b6ad8fed02f7eafc82a8b23c5ebb6525ff8615bf3a95e3e not found: ID does not exist" Jul 23 04:42:23.195940 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.195730 2562 scope.go:117] "RemoveContainer" containerID="0c716638ac6e39d4f06f9f2ab725c8590c33e0d2db80f49fc9bc0c93ad6f395c" Jul 23 04:42:23.196071 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.196046 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c716638ac6e39d4f06f9f2ab725c8590c33e0d2db80f49fc9bc0c93ad6f395c"} err="failed to get container status \"0c716638ac6e39d4f06f9f2ab725c8590c33e0d2db80f49fc9bc0c93ad6f395c\": rpc error: code = NotFound desc = could not find container \"0c716638ac6e39d4f06f9f2ab725c8590c33e0d2db80f49fc9bc0c93ad6f395c\": container with ID starting with 0c716638ac6e39d4f06f9f2ab725c8590c33e0d2db80f49fc9bc0c93ad6f395c not found: ID does not exist" Jul 23 04:42:23.196132 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.196075 2562 scope.go:117] "RemoveContainer" containerID="ca5bc1a0f6240e6068f62fa3b8f1be4b8428d222307ecb31ad1709b6e16c0b2e" Jul 23 04:42:23.196435 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.196413 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca5bc1a0f6240e6068f62fa3b8f1be4b8428d222307ecb31ad1709b6e16c0b2e"} err="failed to get container status \"ca5bc1a0f6240e6068f62fa3b8f1be4b8428d222307ecb31ad1709b6e16c0b2e\": rpc error: code = NotFound desc = could not find container \"ca5bc1a0f6240e6068f62fa3b8f1be4b8428d222307ecb31ad1709b6e16c0b2e\": container with ID starting with ca5bc1a0f6240e6068f62fa3b8f1be4b8428d222307ecb31ad1709b6e16c0b2e not found: ID does not exist" Jul 23 04:42:23.196552 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.196437 2562 scope.go:117] "RemoveContainer" containerID="07d023b8c66dfa8fe3f9e65bb03609c82ae8a33be9e2c72f813940b7ba065181" Jul 23 04:42:23.196985 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.196958 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07d023b8c66dfa8fe3f9e65bb03609c82ae8a33be9e2c72f813940b7ba065181"} err="failed to get container status \"07d023b8c66dfa8fe3f9e65bb03609c82ae8a33be9e2c72f813940b7ba065181\": rpc error: code = NotFound desc = could not find container \"07d023b8c66dfa8fe3f9e65bb03609c82ae8a33be9e2c72f813940b7ba065181\": container with ID starting with 07d023b8c66dfa8fe3f9e65bb03609c82ae8a33be9e2c72f813940b7ba065181 not found: ID does not exist" Jul 23 04:42:23.197217 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.196989 2562 scope.go:117] "RemoveContainer" containerID="64cfd4f8db31dfc4c32932396afb36862ad3653104903254fc485711b54fa24d" Jul 23 04:42:23.197217 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.197048 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 23 04:42:23.197351 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.197292 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64cfd4f8db31dfc4c32932396afb36862ad3653104903254fc485711b54fa24d"} err="failed to get container status \"64cfd4f8db31dfc4c32932396afb36862ad3653104903254fc485711b54fa24d\": rpc error: code = NotFound desc = could not find container \"64cfd4f8db31dfc4c32932396afb36862ad3653104903254fc485711b54fa24d\": container with ID starting with 64cfd4f8db31dfc4c32932396afb36862ad3653104903254fc485711b54fa24d not found: ID does not exist" Jul 23 04:42:23.197351 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.197321 2562 scope.go:117] "RemoveContainer" containerID="6bd8ac7c228623e9bfebc9aacf0fc34033feb54058345a945c65bc97dceb8c8c" Jul 23 04:42:23.197639 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.197608 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bd8ac7c228623e9bfebc9aacf0fc34033feb54058345a945c65bc97dceb8c8c"} err="failed to get container status \"6bd8ac7c228623e9bfebc9aacf0fc34033feb54058345a945c65bc97dceb8c8c\": rpc error: code = NotFound desc = could not find container \"6bd8ac7c228623e9bfebc9aacf0fc34033feb54058345a945c65bc97dceb8c8c\": container with ID starting with 6bd8ac7c228623e9bfebc9aacf0fc34033feb54058345a945c65bc97dceb8c8c not found: ID does not exist" Jul 23 04:42:23.197639 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.197634 2562 scope.go:117] "RemoveContainer" containerID="09c71d41de412cc828294002a80e3c21f19dc9ab558f9b783a20a490e85fba4e" Jul 23 04:42:23.198251 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.198225 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09c71d41de412cc828294002a80e3c21f19dc9ab558f9b783a20a490e85fba4e"} err="failed to get container status \"09c71d41de412cc828294002a80e3c21f19dc9ab558f9b783a20a490e85fba4e\": rpc error: code = NotFound desc = could not find container \"09c71d41de412cc828294002a80e3c21f19dc9ab558f9b783a20a490e85fba4e\": container with ID starting with 09c71d41de412cc828294002a80e3c21f19dc9ab558f9b783a20a490e85fba4e not found: ID does not exist" Jul 23 04:42:23.198251 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.198253 2562 scope.go:117] "RemoveContainer" containerID="cb4426a1ff8c60fb8b6ad8fed02f7eafc82a8b23c5ebb6525ff8615bf3a95e3e" Jul 23 04:42:23.198486 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.198470 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb4426a1ff8c60fb8b6ad8fed02f7eafc82a8b23c5ebb6525ff8615bf3a95e3e"} err="failed to get container status \"cb4426a1ff8c60fb8b6ad8fed02f7eafc82a8b23c5ebb6525ff8615bf3a95e3e\": rpc error: code = NotFound desc = could not find container \"cb4426a1ff8c60fb8b6ad8fed02f7eafc82a8b23c5ebb6525ff8615bf3a95e3e\": container with ID starting with cb4426a1ff8c60fb8b6ad8fed02f7eafc82a8b23c5ebb6525ff8615bf3a95e3e not found: ID does not exist" Jul 23 04:42:23.198536 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.198486 2562 scope.go:117] "RemoveContainer" containerID="0c716638ac6e39d4f06f9f2ab725c8590c33e0d2db80f49fc9bc0c93ad6f395c" Jul 23 04:42:23.198744 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.198697 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c716638ac6e39d4f06f9f2ab725c8590c33e0d2db80f49fc9bc0c93ad6f395c"} err="failed to get container status \"0c716638ac6e39d4f06f9f2ab725c8590c33e0d2db80f49fc9bc0c93ad6f395c\": rpc error: code = NotFound desc = could not find container \"0c716638ac6e39d4f06f9f2ab725c8590c33e0d2db80f49fc9bc0c93ad6f395c\": container with ID starting with 0c716638ac6e39d4f06f9f2ab725c8590c33e0d2db80f49fc9bc0c93ad6f395c not found: ID does not exist" Jul 23 04:42:23.198744 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.198719 2562 scope.go:117] "RemoveContainer" containerID="ca5bc1a0f6240e6068f62fa3b8f1be4b8428d222307ecb31ad1709b6e16c0b2e" Jul 23 04:42:23.199080 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.199054 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca5bc1a0f6240e6068f62fa3b8f1be4b8428d222307ecb31ad1709b6e16c0b2e"} err="failed to get container status \"ca5bc1a0f6240e6068f62fa3b8f1be4b8428d222307ecb31ad1709b6e16c0b2e\": rpc error: code = NotFound desc = could not find container \"ca5bc1a0f6240e6068f62fa3b8f1be4b8428d222307ecb31ad1709b6e16c0b2e\": container with ID starting with ca5bc1a0f6240e6068f62fa3b8f1be4b8428d222307ecb31ad1709b6e16c0b2e not found: ID does not exist" Jul 23 04:42:23.199207 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.199080 2562 scope.go:117] "RemoveContainer" containerID="07d023b8c66dfa8fe3f9e65bb03609c82ae8a33be9e2c72f813940b7ba065181" Jul 23 04:42:23.199391 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.199371 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07d023b8c66dfa8fe3f9e65bb03609c82ae8a33be9e2c72f813940b7ba065181"} err="failed to get container status \"07d023b8c66dfa8fe3f9e65bb03609c82ae8a33be9e2c72f813940b7ba065181\": rpc error: code = NotFound desc = could not find container \"07d023b8c66dfa8fe3f9e65bb03609c82ae8a33be9e2c72f813940b7ba065181\": container with ID starting with 07d023b8c66dfa8fe3f9e65bb03609c82ae8a33be9e2c72f813940b7ba065181 not found: ID does not exist" Jul 23 04:42:23.199472 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.199392 2562 scope.go:117] "RemoveContainer" containerID="64cfd4f8db31dfc4c32932396afb36862ad3653104903254fc485711b54fa24d" Jul 23 04:42:23.199933 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.199733 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64cfd4f8db31dfc4c32932396afb36862ad3653104903254fc485711b54fa24d"} err="failed to get container status \"64cfd4f8db31dfc4c32932396afb36862ad3653104903254fc485711b54fa24d\": rpc error: code = NotFound desc = could not find container \"64cfd4f8db31dfc4c32932396afb36862ad3653104903254fc485711b54fa24d\": container with ID starting with 64cfd4f8db31dfc4c32932396afb36862ad3653104903254fc485711b54fa24d not found: ID does not exist" Jul 23 04:42:23.199933 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.199757 2562 scope.go:117] "RemoveContainer" containerID="6bd8ac7c228623e9bfebc9aacf0fc34033feb54058345a945c65bc97dceb8c8c" Jul 23 04:42:23.200137 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.200115 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bd8ac7c228623e9bfebc9aacf0fc34033feb54058345a945c65bc97dceb8c8c"} err="failed to get container status \"6bd8ac7c228623e9bfebc9aacf0fc34033feb54058345a945c65bc97dceb8c8c\": rpc error: code = NotFound desc = could not find container \"6bd8ac7c228623e9bfebc9aacf0fc34033feb54058345a945c65bc97dceb8c8c\": container with ID starting with 6bd8ac7c228623e9bfebc9aacf0fc34033feb54058345a945c65bc97dceb8c8c not found: ID does not exist" Jul 23 04:42:23.200195 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.200139 2562 scope.go:117] "RemoveContainer" containerID="09c71d41de412cc828294002a80e3c21f19dc9ab558f9b783a20a490e85fba4e" Jul 23 04:42:23.200427 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.200407 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09c71d41de412cc828294002a80e3c21f19dc9ab558f9b783a20a490e85fba4e"} err="failed to get container status \"09c71d41de412cc828294002a80e3c21f19dc9ab558f9b783a20a490e85fba4e\": rpc error: code = NotFound desc = could not find container \"09c71d41de412cc828294002a80e3c21f19dc9ab558f9b783a20a490e85fba4e\": container with ID starting with 09c71d41de412cc828294002a80e3c21f19dc9ab558f9b783a20a490e85fba4e not found: ID does not exist" Jul 23 04:42:23.200492 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.200429 2562 scope.go:117] "RemoveContainer" containerID="cb4426a1ff8c60fb8b6ad8fed02f7eafc82a8b23c5ebb6525ff8615bf3a95e3e" Jul 23 04:42:23.200681 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.200657 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb4426a1ff8c60fb8b6ad8fed02f7eafc82a8b23c5ebb6525ff8615bf3a95e3e"} err="failed to get container status \"cb4426a1ff8c60fb8b6ad8fed02f7eafc82a8b23c5ebb6525ff8615bf3a95e3e\": rpc error: code = NotFound desc = could not find container \"cb4426a1ff8c60fb8b6ad8fed02f7eafc82a8b23c5ebb6525ff8615bf3a95e3e\": container with ID starting with cb4426a1ff8c60fb8b6ad8fed02f7eafc82a8b23c5ebb6525ff8615bf3a95e3e not found: ID does not exist" Jul 23 04:42:23.200730 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.200684 2562 scope.go:117] "RemoveContainer" containerID="0c716638ac6e39d4f06f9f2ab725c8590c33e0d2db80f49fc9bc0c93ad6f395c" Jul 23 04:42:23.200980 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.200960 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c716638ac6e39d4f06f9f2ab725c8590c33e0d2db80f49fc9bc0c93ad6f395c"} err="failed to get container status \"0c716638ac6e39d4f06f9f2ab725c8590c33e0d2db80f49fc9bc0c93ad6f395c\": rpc error: code = NotFound desc = could not find container \"0c716638ac6e39d4f06f9f2ab725c8590c33e0d2db80f49fc9bc0c93ad6f395c\": container with ID starting with 0c716638ac6e39d4f06f9f2ab725c8590c33e0d2db80f49fc9bc0c93ad6f395c not found: ID does not exist" Jul 23 04:42:23.201054 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.200981 2562 scope.go:117] "RemoveContainer" containerID="ca5bc1a0f6240e6068f62fa3b8f1be4b8428d222307ecb31ad1709b6e16c0b2e" Jul 23 04:42:23.201243 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.201224 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca5bc1a0f6240e6068f62fa3b8f1be4b8428d222307ecb31ad1709b6e16c0b2e"} err="failed to get container status \"ca5bc1a0f6240e6068f62fa3b8f1be4b8428d222307ecb31ad1709b6e16c0b2e\": rpc error: code = NotFound desc = could not find container \"ca5bc1a0f6240e6068f62fa3b8f1be4b8428d222307ecb31ad1709b6e16c0b2e\": container with ID starting with ca5bc1a0f6240e6068f62fa3b8f1be4b8428d222307ecb31ad1709b6e16c0b2e not found: ID does not exist" Jul 23 04:42:23.201339 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.201245 2562 scope.go:117] "RemoveContainer" containerID="07d023b8c66dfa8fe3f9e65bb03609c82ae8a33be9e2c72f813940b7ba065181" Jul 23 04:42:23.201497 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.201479 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07d023b8c66dfa8fe3f9e65bb03609c82ae8a33be9e2c72f813940b7ba065181"} err="failed to get container status \"07d023b8c66dfa8fe3f9e65bb03609c82ae8a33be9e2c72f813940b7ba065181\": rpc error: code = NotFound desc = could not find container \"07d023b8c66dfa8fe3f9e65bb03609c82ae8a33be9e2c72f813940b7ba065181\": container with ID starting with 07d023b8c66dfa8fe3f9e65bb03609c82ae8a33be9e2c72f813940b7ba065181 not found: ID does not exist" Jul 23 04:42:23.201572 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.201499 2562 scope.go:117] "RemoveContainer" containerID="64cfd4f8db31dfc4c32932396afb36862ad3653104903254fc485711b54fa24d" Jul 23 04:42:23.201737 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.201719 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64cfd4f8db31dfc4c32932396afb36862ad3653104903254fc485711b54fa24d"} err="failed to get container status \"64cfd4f8db31dfc4c32932396afb36862ad3653104903254fc485711b54fa24d\": rpc error: code = NotFound desc = could not find container \"64cfd4f8db31dfc4c32932396afb36862ad3653104903254fc485711b54fa24d\": container with ID starting with 64cfd4f8db31dfc4c32932396afb36862ad3653104903254fc485711b54fa24d not found: ID does not exist" Jul 23 04:42:23.201779 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.201739 2562 scope.go:117] "RemoveContainer" containerID="6bd8ac7c228623e9bfebc9aacf0fc34033feb54058345a945c65bc97dceb8c8c" Jul 23 04:42:23.202075 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.202040 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bd8ac7c228623e9bfebc9aacf0fc34033feb54058345a945c65bc97dceb8c8c"} err="failed to get container status \"6bd8ac7c228623e9bfebc9aacf0fc34033feb54058345a945c65bc97dceb8c8c\": rpc error: code = NotFound desc = could not find container \"6bd8ac7c228623e9bfebc9aacf0fc34033feb54058345a945c65bc97dceb8c8c\": container with ID starting with 6bd8ac7c228623e9bfebc9aacf0fc34033feb54058345a945c65bc97dceb8c8c not found: ID does not exist" Jul 23 04:42:23.202143 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.202083 2562 scope.go:117] "RemoveContainer" containerID="09c71d41de412cc828294002a80e3c21f19dc9ab558f9b783a20a490e85fba4e" Jul 23 04:42:23.202421 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.202393 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09c71d41de412cc828294002a80e3c21f19dc9ab558f9b783a20a490e85fba4e"} err="failed to get container status \"09c71d41de412cc828294002a80e3c21f19dc9ab558f9b783a20a490e85fba4e\": rpc error: code = NotFound desc = could not find container \"09c71d41de412cc828294002a80e3c21f19dc9ab558f9b783a20a490e85fba4e\": container with ID starting with 09c71d41de412cc828294002a80e3c21f19dc9ab558f9b783a20a490e85fba4e not found: ID does not exist" Jul 23 04:42:23.202421 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.202420 2562 scope.go:117] "RemoveContainer" containerID="cb4426a1ff8c60fb8b6ad8fed02f7eafc82a8b23c5ebb6525ff8615bf3a95e3e" Jul 23 04:42:23.202758 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.202652 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb4426a1ff8c60fb8b6ad8fed02f7eafc82a8b23c5ebb6525ff8615bf3a95e3e"} err="failed to get container status \"cb4426a1ff8c60fb8b6ad8fed02f7eafc82a8b23c5ebb6525ff8615bf3a95e3e\": rpc error: code = NotFound desc = could not find container \"cb4426a1ff8c60fb8b6ad8fed02f7eafc82a8b23c5ebb6525ff8615bf3a95e3e\": container with ID starting with cb4426a1ff8c60fb8b6ad8fed02f7eafc82a8b23c5ebb6525ff8615bf3a95e3e not found: ID does not exist" Jul 23 04:42:23.202758 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.202697 2562 scope.go:117] "RemoveContainer" containerID="0c716638ac6e39d4f06f9f2ab725c8590c33e0d2db80f49fc9bc0c93ad6f395c" Jul 23 04:42:23.203020 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.203000 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c716638ac6e39d4f06f9f2ab725c8590c33e0d2db80f49fc9bc0c93ad6f395c"} err="failed to get container status \"0c716638ac6e39d4f06f9f2ab725c8590c33e0d2db80f49fc9bc0c93ad6f395c\": rpc error: code = NotFound desc = could not find container \"0c716638ac6e39d4f06f9f2ab725c8590c33e0d2db80f49fc9bc0c93ad6f395c\": container with ID starting with 0c716638ac6e39d4f06f9f2ab725c8590c33e0d2db80f49fc9bc0c93ad6f395c not found: ID does not exist" Jul 23 04:42:23.203020 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.203020 2562 scope.go:117] "RemoveContainer" containerID="ca5bc1a0f6240e6068f62fa3b8f1be4b8428d222307ecb31ad1709b6e16c0b2e" Jul 23 04:42:23.203332 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.203295 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca5bc1a0f6240e6068f62fa3b8f1be4b8428d222307ecb31ad1709b6e16c0b2e"} err="failed to get container status \"ca5bc1a0f6240e6068f62fa3b8f1be4b8428d222307ecb31ad1709b6e16c0b2e\": rpc error: code = NotFound desc = could not find container \"ca5bc1a0f6240e6068f62fa3b8f1be4b8428d222307ecb31ad1709b6e16c0b2e\": container with ID starting with ca5bc1a0f6240e6068f62fa3b8f1be4b8428d222307ecb31ad1709b6e16c0b2e not found: ID does not exist" Jul 23 04:42:23.203419 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.203334 2562 scope.go:117] "RemoveContainer" containerID="07d023b8c66dfa8fe3f9e65bb03609c82ae8a33be9e2c72f813940b7ba065181" Jul 23 04:42:23.203630 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.203604 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07d023b8c66dfa8fe3f9e65bb03609c82ae8a33be9e2c72f813940b7ba065181"} err="failed to get container status \"07d023b8c66dfa8fe3f9e65bb03609c82ae8a33be9e2c72f813940b7ba065181\": rpc error: code = NotFound desc = could not find container \"07d023b8c66dfa8fe3f9e65bb03609c82ae8a33be9e2c72f813940b7ba065181\": container with ID starting with 07d023b8c66dfa8fe3f9e65bb03609c82ae8a33be9e2c72f813940b7ba065181 not found: ID does not exist" Jul 23 04:42:23.203630 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.203630 2562 scope.go:117] "RemoveContainer" containerID="64cfd4f8db31dfc4c32932396afb36862ad3653104903254fc485711b54fa24d" Jul 23 04:42:23.203978 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.203923 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64cfd4f8db31dfc4c32932396afb36862ad3653104903254fc485711b54fa24d"} err="failed to get container status \"64cfd4f8db31dfc4c32932396afb36862ad3653104903254fc485711b54fa24d\": rpc error: code = NotFound desc = could not find container \"64cfd4f8db31dfc4c32932396afb36862ad3653104903254fc485711b54fa24d\": container with ID starting with 64cfd4f8db31dfc4c32932396afb36862ad3653104903254fc485711b54fa24d not found: ID does not exist" Jul 23 04:42:23.203978 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.203945 2562 scope.go:117] "RemoveContainer" containerID="6bd8ac7c228623e9bfebc9aacf0fc34033feb54058345a945c65bc97dceb8c8c" Jul 23 04:42:23.204365 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.204335 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bd8ac7c228623e9bfebc9aacf0fc34033feb54058345a945c65bc97dceb8c8c"} err="failed to get container status \"6bd8ac7c228623e9bfebc9aacf0fc34033feb54058345a945c65bc97dceb8c8c\": rpc error: code = NotFound desc = could not find container \"6bd8ac7c228623e9bfebc9aacf0fc34033feb54058345a945c65bc97dceb8c8c\": container with ID starting with 6bd8ac7c228623e9bfebc9aacf0fc34033feb54058345a945c65bc97dceb8c8c not found: ID does not exist" Jul 23 04:42:23.204365 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.204355 2562 scope.go:117] "RemoveContainer" containerID="09c71d41de412cc828294002a80e3c21f19dc9ab558f9b783a20a490e85fba4e" Jul 23 04:42:23.204591 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.204570 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09c71d41de412cc828294002a80e3c21f19dc9ab558f9b783a20a490e85fba4e"} err="failed to get container status \"09c71d41de412cc828294002a80e3c21f19dc9ab558f9b783a20a490e85fba4e\": rpc error: code = NotFound desc = could not find container \"09c71d41de412cc828294002a80e3c21f19dc9ab558f9b783a20a490e85fba4e\": container with ID starting with 09c71d41de412cc828294002a80e3c21f19dc9ab558f9b783a20a490e85fba4e not found: ID does not exist" Jul 23 04:42:23.204671 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.204591 2562 scope.go:117] "RemoveContainer" containerID="cb4426a1ff8c60fb8b6ad8fed02f7eafc82a8b23c5ebb6525ff8615bf3a95e3e" Jul 23 04:42:23.204876 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.204853 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb4426a1ff8c60fb8b6ad8fed02f7eafc82a8b23c5ebb6525ff8615bf3a95e3e"} err="failed to get container status \"cb4426a1ff8c60fb8b6ad8fed02f7eafc82a8b23c5ebb6525ff8615bf3a95e3e\": rpc error: code = NotFound desc = could not find container \"cb4426a1ff8c60fb8b6ad8fed02f7eafc82a8b23c5ebb6525ff8615bf3a95e3e\": container with ID starting with cb4426a1ff8c60fb8b6ad8fed02f7eafc82a8b23c5ebb6525ff8615bf3a95e3e not found: ID does not exist" Jul 23 04:42:23.204930 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.204877 2562 scope.go:117] "RemoveContainer" containerID="0c716638ac6e39d4f06f9f2ab725c8590c33e0d2db80f49fc9bc0c93ad6f395c" Jul 23 04:42:23.205137 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.205117 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c716638ac6e39d4f06f9f2ab725c8590c33e0d2db80f49fc9bc0c93ad6f395c"} err="failed to get container status \"0c716638ac6e39d4f06f9f2ab725c8590c33e0d2db80f49fc9bc0c93ad6f395c\": rpc error: code = NotFound desc = could not find container \"0c716638ac6e39d4f06f9f2ab725c8590c33e0d2db80f49fc9bc0c93ad6f395c\": container with ID starting with 0c716638ac6e39d4f06f9f2ab725c8590c33e0d2db80f49fc9bc0c93ad6f395c not found: ID does not exist" Jul 23 04:42:23.205137 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.205137 2562 scope.go:117] "RemoveContainer" containerID="ca5bc1a0f6240e6068f62fa3b8f1be4b8428d222307ecb31ad1709b6e16c0b2e" Jul 23 04:42:23.205475 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.205427 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca5bc1a0f6240e6068f62fa3b8f1be4b8428d222307ecb31ad1709b6e16c0b2e"} err="failed to get container status \"ca5bc1a0f6240e6068f62fa3b8f1be4b8428d222307ecb31ad1709b6e16c0b2e\": rpc error: code = NotFound desc = could not find container \"ca5bc1a0f6240e6068f62fa3b8f1be4b8428d222307ecb31ad1709b6e16c0b2e\": container with ID starting with ca5bc1a0f6240e6068f62fa3b8f1be4b8428d222307ecb31ad1709b6e16c0b2e not found: ID does not exist" Jul 23 04:42:23.205475 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.205452 2562 scope.go:117] "RemoveContainer" containerID="07d023b8c66dfa8fe3f9e65bb03609c82ae8a33be9e2c72f813940b7ba065181" Jul 23 04:42:23.205693 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.205677 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07d023b8c66dfa8fe3f9e65bb03609c82ae8a33be9e2c72f813940b7ba065181"} err="failed to get container status \"07d023b8c66dfa8fe3f9e65bb03609c82ae8a33be9e2c72f813940b7ba065181\": rpc error: code = NotFound desc = could not find container \"07d023b8c66dfa8fe3f9e65bb03609c82ae8a33be9e2c72f813940b7ba065181\": container with ID starting with 07d023b8c66dfa8fe3f9e65bb03609c82ae8a33be9e2c72f813940b7ba065181 not found: ID does not exist" Jul 23 04:42:23.205737 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.205694 2562 scope.go:117] "RemoveContainer" containerID="64cfd4f8db31dfc4c32932396afb36862ad3653104903254fc485711b54fa24d" Jul 23 04:42:23.205933 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.205911 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64cfd4f8db31dfc4c32932396afb36862ad3653104903254fc485711b54fa24d"} err="failed to get container status \"64cfd4f8db31dfc4c32932396afb36862ad3653104903254fc485711b54fa24d\": rpc error: code = NotFound desc = could not find container \"64cfd4f8db31dfc4c32932396afb36862ad3653104903254fc485711b54fa24d\": container with ID starting with 64cfd4f8db31dfc4c32932396afb36862ad3653104903254fc485711b54fa24d not found: ID does not exist" Jul 23 04:42:23.205933 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.205931 2562 scope.go:117] "RemoveContainer" containerID="6bd8ac7c228623e9bfebc9aacf0fc34033feb54058345a945c65bc97dceb8c8c" Jul 23 04:42:23.206178 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.206157 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bd8ac7c228623e9bfebc9aacf0fc34033feb54058345a945c65bc97dceb8c8c"} err="failed to get container status \"6bd8ac7c228623e9bfebc9aacf0fc34033feb54058345a945c65bc97dceb8c8c\": rpc error: code = NotFound desc = could not find container \"6bd8ac7c228623e9bfebc9aacf0fc34033feb54058345a945c65bc97dceb8c8c\": container with ID starting with 6bd8ac7c228623e9bfebc9aacf0fc34033feb54058345a945c65bc97dceb8c8c not found: ID does not exist" Jul 23 04:42:23.206252 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.206179 2562 scope.go:117] "RemoveContainer" containerID="09c71d41de412cc828294002a80e3c21f19dc9ab558f9b783a20a490e85fba4e" Jul 23 04:42:23.206389 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.206373 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09c71d41de412cc828294002a80e3c21f19dc9ab558f9b783a20a490e85fba4e"} err="failed to get container status \"09c71d41de412cc828294002a80e3c21f19dc9ab558f9b783a20a490e85fba4e\": rpc error: code = NotFound desc = could not find container \"09c71d41de412cc828294002a80e3c21f19dc9ab558f9b783a20a490e85fba4e\": container with ID starting with 09c71d41de412cc828294002a80e3c21f19dc9ab558f9b783a20a490e85fba4e not found: ID does not exist" Jul 23 04:42:23.208077 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.208059 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3cd8820-6afb-4e8e-b2e4-6471f66d12dd" path="/var/lib/kubelet/pods/f3cd8820-6afb-4e8e-b2e4-6471f66d12dd/volumes" Jul 23 04:42:23.208583 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.208571 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 23 04:42:23.261868 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.261817 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7ace0723-60be-4fd5-8251-2706d257e9f9-config\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.262042 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.261946 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7ace0723-60be-4fd5-8251-2706d257e9f9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.262042 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.261973 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7ace0723-60be-4fd5-8251-2706d257e9f9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.262042 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.262026 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7ace0723-60be-4fd5-8251-2706d257e9f9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.262154 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.262092 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmvdf\" (UniqueName: \"kubernetes.io/projected/7ace0723-60be-4fd5-8251-2706d257e9f9-kube-api-access-lmvdf\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.262154 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.262119 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ace0723-60be-4fd5-8251-2706d257e9f9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.262222 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.262193 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7ace0723-60be-4fd5-8251-2706d257e9f9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.262279 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.262263 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7ace0723-60be-4fd5-8251-2706d257e9f9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.262325 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.262288 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7ace0723-60be-4fd5-8251-2706d257e9f9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.262501 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.262484 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7ace0723-60be-4fd5-8251-2706d257e9f9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.262546 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.262515 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7ace0723-60be-4fd5-8251-2706d257e9f9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.262636 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.262616 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7ace0723-60be-4fd5-8251-2706d257e9f9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.262695 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.262653 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7ace0723-60be-4fd5-8251-2706d257e9f9-web-config\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.262695 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.262684 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7ace0723-60be-4fd5-8251-2706d257e9f9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.262790 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.262709 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ace0723-60be-4fd5-8251-2706d257e9f9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.262891 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.262873 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7ace0723-60be-4fd5-8251-2706d257e9f9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.262966 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.262905 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7ace0723-60be-4fd5-8251-2706d257e9f9-config-out\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.262966 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.262921 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7ace0723-60be-4fd5-8251-2706d257e9f9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.262966 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.262938 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ace0723-60be-4fd5-8251-2706d257e9f9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.263066 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.262975 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ace0723-60be-4fd5-8251-2706d257e9f9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.363597 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.363565 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7ace0723-60be-4fd5-8251-2706d257e9f9-config-out\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.364110 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.363600 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7ace0723-60be-4fd5-8251-2706d257e9f9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.364110 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.363618 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ace0723-60be-4fd5-8251-2706d257e9f9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.364110 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.363635 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ace0723-60be-4fd5-8251-2706d257e9f9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.364110 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.363657 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7ace0723-60be-4fd5-8251-2706d257e9f9-config\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.364110 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.363774 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7ace0723-60be-4fd5-8251-2706d257e9f9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.364110 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.363803 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7ace0723-60be-4fd5-8251-2706d257e9f9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.364110 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.363868 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7ace0723-60be-4fd5-8251-2706d257e9f9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.364110 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.363907 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmvdf\" (UniqueName: \"kubernetes.io/projected/7ace0723-60be-4fd5-8251-2706d257e9f9-kube-api-access-lmvdf\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.364110 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.363938 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ace0723-60be-4fd5-8251-2706d257e9f9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.364110 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.363976 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7ace0723-60be-4fd5-8251-2706d257e9f9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.364110 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.364019 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7ace0723-60be-4fd5-8251-2706d257e9f9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.364110 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.364045 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7ace0723-60be-4fd5-8251-2706d257e9f9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.364110 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.364071 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7ace0723-60be-4fd5-8251-2706d257e9f9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.364110 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.364103 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7ace0723-60be-4fd5-8251-2706d257e9f9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.364990 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.364171 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7ace0723-60be-4fd5-8251-2706d257e9f9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.364990 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.364196 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7ace0723-60be-4fd5-8251-2706d257e9f9-web-config\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.364990 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.364220 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7ace0723-60be-4fd5-8251-2706d257e9f9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.364990 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.364246 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ace0723-60be-4fd5-8251-2706d257e9f9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.364990 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.364273 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7ace0723-60be-4fd5-8251-2706d257e9f9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.364990 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.364449 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ace0723-60be-4fd5-8251-2706d257e9f9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.364990 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.364722 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7ace0723-60be-4fd5-8251-2706d257e9f9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.365342 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.364999 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ace0723-60be-4fd5-8251-2706d257e9f9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.365342 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.365290 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ace0723-60be-4fd5-8251-2706d257e9f9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.365857 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.365816 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7ace0723-60be-4fd5-8251-2706d257e9f9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.367190 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.366322 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7ace0723-60be-4fd5-8251-2706d257e9f9-config-out\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.367190 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.366802 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/7ace0723-60be-4fd5-8251-2706d257e9f9-config\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.368303 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.367382 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7ace0723-60be-4fd5-8251-2706d257e9f9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.368303 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.367666 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7ace0723-60be-4fd5-8251-2706d257e9f9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.368303 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.368029 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7ace0723-60be-4fd5-8251-2706d257e9f9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.369576 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.368812 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ace0723-60be-4fd5-8251-2706d257e9f9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.369576 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.369513 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7ace0723-60be-4fd5-8251-2706d257e9f9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.369730 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.369636 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7ace0723-60be-4fd5-8251-2706d257e9f9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.369730 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.369661 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7ace0723-60be-4fd5-8251-2706d257e9f9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.369876 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.369846 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7ace0723-60be-4fd5-8251-2706d257e9f9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.370275 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.370248 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7ace0723-60be-4fd5-8251-2706d257e9f9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.370370 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.370290 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7ace0723-60be-4fd5-8251-2706d257e9f9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.370411 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.370383 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7ace0723-60be-4fd5-8251-2706d257e9f9-web-config\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.370777 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.370750 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7ace0723-60be-4fd5-8251-2706d257e9f9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.376346 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.376318 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmvdf\" (UniqueName: \"kubernetes.io/projected/7ace0723-60be-4fd5-8251-2706d257e9f9-kube-api-access-lmvdf\") pod \"prometheus-k8s-0\" (UID: \"7ace0723-60be-4fd5-8251-2706d257e9f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.498218 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.498175 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:23.642052 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:23.642023 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 23 04:42:23.643726 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:42:23.643700 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ace0723_60be_4fd5_8251_2706d257e9f9.slice/crio-7ac5dcdcbdaf42ca62a2b25448c6bc6981a1001fe44d25ca8a7eaa40ed1844c4 WatchSource:0}: Error finding container 7ac5dcdcbdaf42ca62a2b25448c6bc6981a1001fe44d25ca8a7eaa40ed1844c4: Status 404 returned error can't find the container with id 7ac5dcdcbdaf42ca62a2b25448c6bc6981a1001fe44d25ca8a7eaa40ed1844c4 Jul 23 04:42:24.127368 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:24.127335 2562 generic.go:356] "Generic (PLEG): container finished" podID="7ace0723-60be-4fd5-8251-2706d257e9f9" containerID="0429fea04b832d96b1f9dd5045766094d59a93c6309beeafc536bec276b462a9" exitCode=0 Jul 23 04:42:24.127524 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:24.127422 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ace0723-60be-4fd5-8251-2706d257e9f9","Type":"ContainerDied","Data":"0429fea04b832d96b1f9dd5045766094d59a93c6309beeafc536bec276b462a9"} Jul 23 04:42:24.127524 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:24.127490 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ace0723-60be-4fd5-8251-2706d257e9f9","Type":"ContainerStarted","Data":"7ac5dcdcbdaf42ca62a2b25448c6bc6981a1001fe44d25ca8a7eaa40ed1844c4"} Jul 23 04:42:25.134334 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:25.134300 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ace0723-60be-4fd5-8251-2706d257e9f9","Type":"ContainerStarted","Data":"7ad7c99ca8e741f1cb19e844f03572ccac49553308f0f37f17cb5c104a74ae7f"} Jul 23 04:42:25.134654 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:25.134391 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ace0723-60be-4fd5-8251-2706d257e9f9","Type":"ContainerStarted","Data":"77a3bebaadd88a99a6c2c2d36bd2b5d166083e7cd9d66e2857c0788974057bca"} Jul 23 04:42:25.134654 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:25.134420 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ace0723-60be-4fd5-8251-2706d257e9f9","Type":"ContainerStarted","Data":"0eed755e4b1c7f7817f83a4c3d1a214f3904c8f9c69b93aefd8233dc81a98479"} Jul 23 04:42:25.134654 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:25.134444 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ace0723-60be-4fd5-8251-2706d257e9f9","Type":"ContainerStarted","Data":"fc50630e6b0264599faead4bce57400451b84fc369aeee80db21007c5e68005c"} Jul 23 04:42:25.134654 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:25.134479 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ace0723-60be-4fd5-8251-2706d257e9f9","Type":"ContainerStarted","Data":"d529b8a4948eb3f48ab79d9407b0669d1571fc7471088e2abe5080f8eb33d9ee"} Jul 23 04:42:25.134654 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:25.134504 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ace0723-60be-4fd5-8251-2706d257e9f9","Type":"ContainerStarted","Data":"925fda001385a3f82cf1ed4f9d9373d309f44dca93e4f4bba50333bcbde20e5c"} Jul 23 04:42:25.167220 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:25.167173 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.146948345 podStartE2EDuration="2.167157628s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.209283ms" isStatefulPod=false podCreationTimestamp="2026-07-23 04:42:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:42:25.164678193 +0000 UTC m=+180.507191080" watchObservedRunningTime="2026-07-23 04:42:25.167157628 +0000 UTC m=+180.509670554" Jul 23 04:42:28.498955 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:28.498903 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:42:37.160913 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:42:37.160878 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c6dc857b6-2dkkn"] Jul 23 04:43:02.181998 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:43:02.181953 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-c6dc857b6-2dkkn" podUID="777a4f89-ebcc-490a-bd94-6d07a9bb871c" containerName="console" containerID="cri-o://e0b93e0911b74e6711b1602ace57236ff040e4b91f5eb29147ef40511fec33aa" gracePeriod=15 Jul 23 04:43:02.420453 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:43:02.420430 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c6dc857b6-2dkkn" Jul 23 04:43:02.506003 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:43:02.505927 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/777a4f89-ebcc-490a-bd94-6d07a9bb871c-console-config\") pod \"777a4f89-ebcc-490a-bd94-6d07a9bb871c\" (UID: \"777a4f89-ebcc-490a-bd94-6d07a9bb871c\") " Jul 23 04:43:02.506003 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:43:02.505973 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/777a4f89-ebcc-490a-bd94-6d07a9bb871c-oauth-serving-cert\") pod \"777a4f89-ebcc-490a-bd94-6d07a9bb871c\" (UID: \"777a4f89-ebcc-490a-bd94-6d07a9bb871c\") " Jul 23 04:43:02.506210 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:43:02.506016 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/777a4f89-ebcc-490a-bd94-6d07a9bb871c-service-ca\") pod \"777a4f89-ebcc-490a-bd94-6d07a9bb871c\" (UID: \"777a4f89-ebcc-490a-bd94-6d07a9bb871c\") " Jul 23 04:43:02.506210 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:43:02.506054 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98zxp\" (UniqueName: \"kubernetes.io/projected/777a4f89-ebcc-490a-bd94-6d07a9bb871c-kube-api-access-98zxp\") pod \"777a4f89-ebcc-490a-bd94-6d07a9bb871c\" (UID: \"777a4f89-ebcc-490a-bd94-6d07a9bb871c\") " Jul 23 04:43:02.506210 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:43:02.506078 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/777a4f89-ebcc-490a-bd94-6d07a9bb871c-console-serving-cert\") pod \"777a4f89-ebcc-490a-bd94-6d07a9bb871c\" (UID: \"777a4f89-ebcc-490a-bd94-6d07a9bb871c\") " Jul 23 04:43:02.506210 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:43:02.506128 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/777a4f89-ebcc-490a-bd94-6d07a9bb871c-trusted-ca-bundle\") pod \"777a4f89-ebcc-490a-bd94-6d07a9bb871c\" (UID: \"777a4f89-ebcc-490a-bd94-6d07a9bb871c\") " Jul 23 04:43:02.506210 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:43:02.506159 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/777a4f89-ebcc-490a-bd94-6d07a9bb871c-console-oauth-config\") pod \"777a4f89-ebcc-490a-bd94-6d07a9bb871c\" (UID: \"777a4f89-ebcc-490a-bd94-6d07a9bb871c\") " Jul 23 04:43:02.506434 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:43:02.506357 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/777a4f89-ebcc-490a-bd94-6d07a9bb871c-console-config" (OuterVolumeSpecName: "console-config") pod "777a4f89-ebcc-490a-bd94-6d07a9bb871c" (UID: "777a4f89-ebcc-490a-bd94-6d07a9bb871c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 04:43:02.506434 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:43:02.506380 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/777a4f89-ebcc-490a-bd94-6d07a9bb871c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "777a4f89-ebcc-490a-bd94-6d07a9bb871c" (UID: "777a4f89-ebcc-490a-bd94-6d07a9bb871c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 04:43:02.506541 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:43:02.506443 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/777a4f89-ebcc-490a-bd94-6d07a9bb871c-console-config\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:43:02.506541 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:43:02.506461 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/777a4f89-ebcc-490a-bd94-6d07a9bb871c-oauth-serving-cert\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:43:02.506682 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:43:02.506647 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/777a4f89-ebcc-490a-bd94-6d07a9bb871c-service-ca" (OuterVolumeSpecName: "service-ca") pod "777a4f89-ebcc-490a-bd94-6d07a9bb871c" (UID: "777a4f89-ebcc-490a-bd94-6d07a9bb871c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 04:43:02.506759 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:43:02.506728 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/777a4f89-ebcc-490a-bd94-6d07a9bb871c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "777a4f89-ebcc-490a-bd94-6d07a9bb871c" (UID: "777a4f89-ebcc-490a-bd94-6d07a9bb871c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 04:43:02.508745 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:43:02.508719 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/777a4f89-ebcc-490a-bd94-6d07a9bb871c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "777a4f89-ebcc-490a-bd94-6d07a9bb871c" (UID: "777a4f89-ebcc-490a-bd94-6d07a9bb871c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 04:43:02.509354 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:43:02.509328 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/777a4f89-ebcc-490a-bd94-6d07a9bb871c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "777a4f89-ebcc-490a-bd94-6d07a9bb871c" (UID: "777a4f89-ebcc-490a-bd94-6d07a9bb871c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 04:43:02.509434 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:43:02.509365 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/777a4f89-ebcc-490a-bd94-6d07a9bb871c-kube-api-access-98zxp" (OuterVolumeSpecName: "kube-api-access-98zxp") pod "777a4f89-ebcc-490a-bd94-6d07a9bb871c" (UID: "777a4f89-ebcc-490a-bd94-6d07a9bb871c"). InnerVolumeSpecName "kube-api-access-98zxp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 04:43:02.607884 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:43:02.607826 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/777a4f89-ebcc-490a-bd94-6d07a9bb871c-trusted-ca-bundle\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:43:02.607884 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:43:02.607881 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/777a4f89-ebcc-490a-bd94-6d07a9bb871c-console-oauth-config\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:43:02.608042 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:43:02.607892 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/777a4f89-ebcc-490a-bd94-6d07a9bb871c-service-ca\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:43:02.608042 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:43:02.607901 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98zxp\" (UniqueName: \"kubernetes.io/projected/777a4f89-ebcc-490a-bd94-6d07a9bb871c-kube-api-access-98zxp\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:43:02.608042 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:43:02.607909 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/777a4f89-ebcc-490a-bd94-6d07a9bb871c-console-serving-cert\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:43:03.285100 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:43:03.285066 2562 generic.go:356] "Generic (PLEG): container finished" podID="777a4f89-ebcc-490a-bd94-6d07a9bb871c" containerID="e0b93e0911b74e6711b1602ace57236ff040e4b91f5eb29147ef40511fec33aa" exitCode=0 Jul 23 04:43:03.285507 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:43:03.285115 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c6dc857b6-2dkkn" Jul 23 04:43:03.285507 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:43:03.285131 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c6dc857b6-2dkkn" event={"ID":"777a4f89-ebcc-490a-bd94-6d07a9bb871c","Type":"ContainerDied","Data":"e0b93e0911b74e6711b1602ace57236ff040e4b91f5eb29147ef40511fec33aa"} Jul 23 04:43:03.285507 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:43:03.285172 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c6dc857b6-2dkkn" event={"ID":"777a4f89-ebcc-490a-bd94-6d07a9bb871c","Type":"ContainerDied","Data":"aeef1064a92d1170de0a28892cd267137ec12c37ce00795c3c2b4cb0eb603142"} Jul 23 04:43:03.285507 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:43:03.285191 2562 scope.go:117] "RemoveContainer" containerID="e0b93e0911b74e6711b1602ace57236ff040e4b91f5eb29147ef40511fec33aa" Jul 23 04:43:03.293557 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:43:03.293537 2562 scope.go:117] "RemoveContainer" containerID="e0b93e0911b74e6711b1602ace57236ff040e4b91f5eb29147ef40511fec33aa" Jul 23 04:43:03.293881 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:43:03.293860 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0b93e0911b74e6711b1602ace57236ff040e4b91f5eb29147ef40511fec33aa\": container with ID starting with e0b93e0911b74e6711b1602ace57236ff040e4b91f5eb29147ef40511fec33aa not found: ID does not exist" containerID="e0b93e0911b74e6711b1602ace57236ff040e4b91f5eb29147ef40511fec33aa" Jul 23 04:43:03.293964 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:43:03.293892 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0b93e0911b74e6711b1602ace57236ff040e4b91f5eb29147ef40511fec33aa"} err="failed to get container status \"e0b93e0911b74e6711b1602ace57236ff040e4b91f5eb29147ef40511fec33aa\": rpc error: code = NotFound desc = could not find container \"e0b93e0911b74e6711b1602ace57236ff040e4b91f5eb29147ef40511fec33aa\": container with ID starting with e0b93e0911b74e6711b1602ace57236ff040e4b91f5eb29147ef40511fec33aa not found: ID does not exist" Jul 23 04:43:03.303880 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:43:03.303859 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c6dc857b6-2dkkn"] Jul 23 04:43:03.308665 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:43:03.308589 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-c6dc857b6-2dkkn"] Jul 23 04:43:05.206194 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:43:05.206166 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="777a4f89-ebcc-490a-bd94-6d07a9bb871c" path="/var/lib/kubelet/pods/777a4f89-ebcc-490a-bd94-6d07a9bb871c/volumes" Jul 23 04:43:23.499178 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:43:23.499132 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:43:23.515435 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:43:23.515404 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:43:24.384765 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:43:24.384737 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 04:44:25.070596 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:44:25.070565 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vphbq_b3007292-2534-4dc0-82a3-41d6a7d34c0c/ovn-acl-logging/0.log" Jul 23 04:44:25.078695 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:44:25.078670 2562 kubelet.go:1648] "Image garbage collection succeeded" Jul 23 04:44:25.079134 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:44:25.079116 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vphbq_b3007292-2534-4dc0-82a3-41d6a7d34c0c/ovn-acl-logging/0.log" Jul 23 04:45:49.337343 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.337314 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6576c599b-jw2bk"] Jul 23 04:45:49.337949 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.337905 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="777a4f89-ebcc-490a-bd94-6d07a9bb871c" containerName="console" Jul 23 04:45:49.337949 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.337924 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="777a4f89-ebcc-490a-bd94-6d07a9bb871c" containerName="console" Jul 23 04:45:49.338082 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.338030 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="777a4f89-ebcc-490a-bd94-6d07a9bb871c" containerName="console" Jul 23 04:45:49.340963 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.340943 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6576c599b-jw2bk" Jul 23 04:45:49.343772 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.343748 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-pzdj8\"" Jul 23 04:45:49.344036 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.344020 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 23 04:45:49.344441 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.344413 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 23 04:45:49.344680 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.344656 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 23 04:45:49.345042 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.345028 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 23 04:45:49.356270 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.356250 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6576c599b-jw2bk"] Jul 23 04:45:49.379229 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.379203 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4c1ec1df-d68f-4e79-9463-c4a6f58629ec-manifests\") pod \"opendatahub-operator-controller-manager-6576c599b-jw2bk\" (UID: \"4c1ec1df-d68f-4e79-9463-c4a6f58629ec\") " pod="opendatahub/opendatahub-operator-controller-manager-6576c599b-jw2bk" Jul 23 04:45:49.379495 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.379239 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4c1ec1df-d68f-4e79-9463-c4a6f58629ec-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6576c599b-jw2bk\" (UID: \"4c1ec1df-d68f-4e79-9463-c4a6f58629ec\") " pod="opendatahub/opendatahub-operator-controller-manager-6576c599b-jw2bk" Jul 23 04:45:49.379495 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.379279 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c1ec1df-d68f-4e79-9463-c4a6f58629ec-tmp\") pod \"opendatahub-operator-controller-manager-6576c599b-jw2bk\" (UID: \"4c1ec1df-d68f-4e79-9463-c4a6f58629ec\") " pod="opendatahub/opendatahub-operator-controller-manager-6576c599b-jw2bk" Jul 23 04:45:49.379495 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.379305 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcl6n\" (UniqueName: \"kubernetes.io/projected/4c1ec1df-d68f-4e79-9463-c4a6f58629ec-kube-api-access-gcl6n\") pod \"opendatahub-operator-controller-manager-6576c599b-jw2bk\" (UID: \"4c1ec1df-d68f-4e79-9463-c4a6f58629ec\") " pod="opendatahub/opendatahub-operator-controller-manager-6576c599b-jw2bk" Jul 23 04:45:49.379495 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.379326 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4c1ec1df-d68f-4e79-9463-c4a6f58629ec-webhook-cert\") pod \"opendatahub-operator-controller-manager-6576c599b-jw2bk\" (UID: \"4c1ec1df-d68f-4e79-9463-c4a6f58629ec\") " pod="opendatahub/opendatahub-operator-controller-manager-6576c599b-jw2bk" Jul 23 04:45:49.479797 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.479752 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c1ec1df-d68f-4e79-9463-c4a6f58629ec-tmp\") pod \"opendatahub-operator-controller-manager-6576c599b-jw2bk\" (UID: \"4c1ec1df-d68f-4e79-9463-c4a6f58629ec\") " pod="opendatahub/opendatahub-operator-controller-manager-6576c599b-jw2bk" Jul 23 04:45:49.479797 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.479795 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcl6n\" (UniqueName: \"kubernetes.io/projected/4c1ec1df-d68f-4e79-9463-c4a6f58629ec-kube-api-access-gcl6n\") pod \"opendatahub-operator-controller-manager-6576c599b-jw2bk\" (UID: \"4c1ec1df-d68f-4e79-9463-c4a6f58629ec\") " pod="opendatahub/opendatahub-operator-controller-manager-6576c599b-jw2bk" Jul 23 04:45:49.479797 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.479817 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4c1ec1df-d68f-4e79-9463-c4a6f58629ec-webhook-cert\") pod \"opendatahub-operator-controller-manager-6576c599b-jw2bk\" (UID: \"4c1ec1df-d68f-4e79-9463-c4a6f58629ec\") " pod="opendatahub/opendatahub-operator-controller-manager-6576c599b-jw2bk" Jul 23 04:45:49.480168 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.479871 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4c1ec1df-d68f-4e79-9463-c4a6f58629ec-manifests\") pod \"opendatahub-operator-controller-manager-6576c599b-jw2bk\" (UID: \"4c1ec1df-d68f-4e79-9463-c4a6f58629ec\") " pod="opendatahub/opendatahub-operator-controller-manager-6576c599b-jw2bk" Jul 23 04:45:49.480168 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.479900 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4c1ec1df-d68f-4e79-9463-c4a6f58629ec-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6576c599b-jw2bk\" (UID: \"4c1ec1df-d68f-4e79-9463-c4a6f58629ec\") " pod="opendatahub/opendatahub-operator-controller-manager-6576c599b-jw2bk" Jul 23 04:45:49.480334 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.480270 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4c1ec1df-d68f-4e79-9463-c4a6f58629ec-manifests\") pod \"opendatahub-operator-controller-manager-6576c599b-jw2bk\" (UID: \"4c1ec1df-d68f-4e79-9463-c4a6f58629ec\") " pod="opendatahub/opendatahub-operator-controller-manager-6576c599b-jw2bk" Jul 23 04:45:49.482362 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.482333 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c1ec1df-d68f-4e79-9463-c4a6f58629ec-tmp\") pod \"opendatahub-operator-controller-manager-6576c599b-jw2bk\" (UID: \"4c1ec1df-d68f-4e79-9463-c4a6f58629ec\") " pod="opendatahub/opendatahub-operator-controller-manager-6576c599b-jw2bk" Jul 23 04:45:49.482535 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.482511 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4c1ec1df-d68f-4e79-9463-c4a6f58629ec-webhook-cert\") pod \"opendatahub-operator-controller-manager-6576c599b-jw2bk\" (UID: \"4c1ec1df-d68f-4e79-9463-c4a6f58629ec\") " pod="opendatahub/opendatahub-operator-controller-manager-6576c599b-jw2bk" Jul 23 04:45:49.482634 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.482589 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4c1ec1df-d68f-4e79-9463-c4a6f58629ec-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6576c599b-jw2bk\" (UID: \"4c1ec1df-d68f-4e79-9463-c4a6f58629ec\") " pod="opendatahub/opendatahub-operator-controller-manager-6576c599b-jw2bk" Jul 23 04:45:49.489913 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.489889 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcl6n\" (UniqueName: \"kubernetes.io/projected/4c1ec1df-d68f-4e79-9463-c4a6f58629ec-kube-api-access-gcl6n\") pod \"opendatahub-operator-controller-manager-6576c599b-jw2bk\" (UID: \"4c1ec1df-d68f-4e79-9463-c4a6f58629ec\") " pod="opendatahub/opendatahub-operator-controller-manager-6576c599b-jw2bk" Jul 23 04:45:49.534790 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.534765 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-56ccd69865-qjbwv"] Jul 23 04:45:49.538210 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.538193 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-56ccd69865-qjbwv" Jul 23 04:45:49.540682 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.540653 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 23 04:45:49.540772 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.540694 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 23 04:45:49.540772 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.540702 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 23 04:45:49.541265 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.541249 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-8jvtk\"" Jul 23 04:45:49.541380 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.541343 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 23 04:45:49.541380 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.541352 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 23 04:45:49.558845 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.558809 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-56ccd69865-qjbwv"] Jul 23 04:45:49.580804 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.580785 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28jbp\" (UniqueName: \"kubernetes.io/projected/18f43204-e1ae-4559-b725-54053a68b3e5-kube-api-access-28jbp\") pod \"lws-controller-manager-56ccd69865-qjbwv\" (UID: \"18f43204-e1ae-4559-b725-54053a68b3e5\") " pod="openshift-lws-operator/lws-controller-manager-56ccd69865-qjbwv" Jul 23 04:45:49.580917 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.580828 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/18f43204-e1ae-4559-b725-54053a68b3e5-manager-config\") pod \"lws-controller-manager-56ccd69865-qjbwv\" (UID: \"18f43204-e1ae-4559-b725-54053a68b3e5\") " pod="openshift-lws-operator/lws-controller-manager-56ccd69865-qjbwv" Jul 23 04:45:49.580917 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.580894 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/18f43204-e1ae-4559-b725-54053a68b3e5-cert\") pod \"lws-controller-manager-56ccd69865-qjbwv\" (UID: \"18f43204-e1ae-4559-b725-54053a68b3e5\") " pod="openshift-lws-operator/lws-controller-manager-56ccd69865-qjbwv" Jul 23 04:45:49.580986 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.580953 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/18f43204-e1ae-4559-b725-54053a68b3e5-metrics-cert\") pod \"lws-controller-manager-56ccd69865-qjbwv\" (UID: \"18f43204-e1ae-4559-b725-54053a68b3e5\") " pod="openshift-lws-operator/lws-controller-manager-56ccd69865-qjbwv" Jul 23 04:45:49.650392 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.650313 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6576c599b-jw2bk" Jul 23 04:45:49.681508 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.681486 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28jbp\" (UniqueName: \"kubernetes.io/projected/18f43204-e1ae-4559-b725-54053a68b3e5-kube-api-access-28jbp\") pod \"lws-controller-manager-56ccd69865-qjbwv\" (UID: \"18f43204-e1ae-4559-b725-54053a68b3e5\") " pod="openshift-lws-operator/lws-controller-manager-56ccd69865-qjbwv" Jul 23 04:45:49.681605 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.681527 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/18f43204-e1ae-4559-b725-54053a68b3e5-manager-config\") pod \"lws-controller-manager-56ccd69865-qjbwv\" (UID: \"18f43204-e1ae-4559-b725-54053a68b3e5\") " pod="openshift-lws-operator/lws-controller-manager-56ccd69865-qjbwv" Jul 23 04:45:49.681605 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.681549 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/18f43204-e1ae-4559-b725-54053a68b3e5-cert\") pod \"lws-controller-manager-56ccd69865-qjbwv\" (UID: \"18f43204-e1ae-4559-b725-54053a68b3e5\") " pod="openshift-lws-operator/lws-controller-manager-56ccd69865-qjbwv" Jul 23 04:45:49.681605 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.681591 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/18f43204-e1ae-4559-b725-54053a68b3e5-metrics-cert\") pod \"lws-controller-manager-56ccd69865-qjbwv\" (UID: \"18f43204-e1ae-4559-b725-54053a68b3e5\") " pod="openshift-lws-operator/lws-controller-manager-56ccd69865-qjbwv" Jul 23 04:45:49.682200 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.682116 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/18f43204-e1ae-4559-b725-54053a68b3e5-manager-config\") pod \"lws-controller-manager-56ccd69865-qjbwv\" (UID: \"18f43204-e1ae-4559-b725-54053a68b3e5\") " pod="openshift-lws-operator/lws-controller-manager-56ccd69865-qjbwv" Jul 23 04:45:49.684083 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.684062 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/18f43204-e1ae-4559-b725-54053a68b3e5-cert\") pod \"lws-controller-manager-56ccd69865-qjbwv\" (UID: \"18f43204-e1ae-4559-b725-54053a68b3e5\") " pod="openshift-lws-operator/lws-controller-manager-56ccd69865-qjbwv" Jul 23 04:45:49.684160 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.684090 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/18f43204-e1ae-4559-b725-54053a68b3e5-metrics-cert\") pod \"lws-controller-manager-56ccd69865-qjbwv\" (UID: \"18f43204-e1ae-4559-b725-54053a68b3e5\") " pod="openshift-lws-operator/lws-controller-manager-56ccd69865-qjbwv" Jul 23 04:45:49.699906 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.699351 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28jbp\" (UniqueName: \"kubernetes.io/projected/18f43204-e1ae-4559-b725-54053a68b3e5-kube-api-access-28jbp\") pod \"lws-controller-manager-56ccd69865-qjbwv\" (UID: \"18f43204-e1ae-4559-b725-54053a68b3e5\") " pod="openshift-lws-operator/lws-controller-manager-56ccd69865-qjbwv" Jul 23 04:45:49.787076 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.787053 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6576c599b-jw2bk"] Jul 23 04:45:49.789580 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:45:49.789555 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c1ec1df_d68f_4e79_9463_c4a6f58629ec.slice/crio-c95e98c51716fe8bf662fbcd4b38518cd9e34dd3b7803966f15068ffdda5038d WatchSource:0}: Error finding container c95e98c51716fe8bf662fbcd4b38518cd9e34dd3b7803966f15068ffdda5038d: Status 404 returned error can't find the container with id c95e98c51716fe8bf662fbcd4b38518cd9e34dd3b7803966f15068ffdda5038d Jul 23 04:45:49.791247 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.791232 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 04:45:49.849658 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.849640 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-56ccd69865-qjbwv" Jul 23 04:45:49.930477 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.930443 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6576c599b-jw2bk" event={"ID":"4c1ec1df-d68f-4e79-9463-c4a6f58629ec","Type":"ContainerStarted","Data":"c95e98c51716fe8bf662fbcd4b38518cd9e34dd3b7803966f15068ffdda5038d"} Jul 23 04:45:49.978359 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:49.978335 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-56ccd69865-qjbwv"] Jul 23 04:45:49.980506 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:45:49.980486 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18f43204_e1ae_4559_b725_54053a68b3e5.slice/crio-4baf6d883b5ed21fd04647517598c315a5eb90f61607e042315c52f5be034ce2 WatchSource:0}: Error finding container 4baf6d883b5ed21fd04647517598c315a5eb90f61607e042315c52f5be034ce2: Status 404 returned error can't find the container with id 4baf6d883b5ed21fd04647517598c315a5eb90f61607e042315c52f5be034ce2 Jul 23 04:45:50.941197 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:50.941161 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-56ccd69865-qjbwv" event={"ID":"18f43204-e1ae-4559-b725-54053a68b3e5","Type":"ContainerStarted","Data":"4baf6d883b5ed21fd04647517598c315a5eb90f61607e042315c52f5be034ce2"} Jul 23 04:45:54.962402 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:54.962363 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6576c599b-jw2bk" event={"ID":"4c1ec1df-d68f-4e79-9463-c4a6f58629ec","Type":"ContainerStarted","Data":"0aeb675c4b374c48f4e66c45babf253adeec9ce526a777b5866b9eebab8cb1cb"} Jul 23 04:45:54.963799 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:54.963776 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-56ccd69865-qjbwv" event={"ID":"18f43204-e1ae-4559-b725-54053a68b3e5","Type":"ContainerStarted","Data":"ab9172006708927e5463e1704d9281ec53f1df677e1127a09fba9e05c433af3a"} Jul 23 04:45:54.963913 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:54.963881 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-56ccd69865-qjbwv" Jul 23 04:45:55.005902 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:55.005856 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-56ccd69865-qjbwv" podStartSLOduration=2.001471721 podStartE2EDuration="6.00582323s" totalImagesPullingTime="4.004351509s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:45:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:45:55.004689646 +0000 UTC m=+390.347202535" watchObservedRunningTime="2026-07-23 04:45:55.00582323 +0000 UTC m=+390.348336115" Jul 23 04:45:55.968979 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:55.968947 2562 generic.go:356] "Generic (PLEG): container finished" podID="4c1ec1df-d68f-4e79-9463-c4a6f58629ec" containerID="0aeb675c4b374c48f4e66c45babf253adeec9ce526a777b5866b9eebab8cb1cb" exitCode=0 Jul 23 04:45:55.969431 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:55.969017 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6576c599b-jw2bk" event={"ID":"4c1ec1df-d68f-4e79-9463-c4a6f58629ec","Type":"ContainerDied","Data":"0aeb675c4b374c48f4e66c45babf253adeec9ce526a777b5866b9eebab8cb1cb"} Jul 23 04:45:56.974754 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:56.974713 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6576c599b-jw2bk" event={"ID":"4c1ec1df-d68f-4e79-9463-c4a6f58629ec","Type":"ContainerStarted","Data":"dc500fd06eca3ddd4cc5d51517203b5b635d25da54edf31dccaab10315f07eca"} Jul 23 04:45:56.975105 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:56.974853 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6576c599b-jw2bk" Jul 23 04:45:57.001545 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:45:57.001447 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6576c599b-jw2bk" podStartSLOduration=2.460871562 podStartE2EDuration="8.00142761s" totalImagesPullingTime="4.536065794s" totalInitContainerRuntime="1.004490254s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:45:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:45:56.999890357 +0000 UTC m=+392.342403234" watchObservedRunningTime="2026-07-23 04:45:57.00142761 +0000 UTC m=+392.343940497" Jul 23 04:46:05.971618 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:46:05.971588 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-56ccd69865-qjbwv" Jul 23 04:46:07.982285 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:46:07.982256 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6576c599b-jw2bk" Jul 23 04:46:20.811899 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:46:20.811868 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7f5c59f4bf-xkxqh"] Jul 23 04:46:20.814294 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:46:20.814277 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7f5c59f4bf-xkxqh" Jul 23 04:46:20.817363 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:46:20.817333 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 23 04:46:20.820039 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:46:20.820018 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 23 04:46:20.820213 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:46:20.820191 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-r2zcj\"" Jul 23 04:46:20.829147 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:46:20.829127 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7f5c59f4bf-xkxqh"] Jul 23 04:46:20.935915 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:46:20.935883 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28e63d2c-7c59-4d3b-bce8-0b2ad7aeaf09-tls-certs\") pod \"kube-auth-proxy-7f5c59f4bf-xkxqh\" (UID: \"28e63d2c-7c59-4d3b-bce8-0b2ad7aeaf09\") " pod="openshift-ingress/kube-auth-proxy-7f5c59f4bf-xkxqh" Jul 23 04:46:20.936057 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:46:20.935927 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/28e63d2c-7c59-4d3b-bce8-0b2ad7aeaf09-tmp\") pod \"kube-auth-proxy-7f5c59f4bf-xkxqh\" (UID: \"28e63d2c-7c59-4d3b-bce8-0b2ad7aeaf09\") " pod="openshift-ingress/kube-auth-proxy-7f5c59f4bf-xkxqh" Jul 23 04:46:20.936057 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:46:20.935997 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77x4k\" (UniqueName: \"kubernetes.io/projected/28e63d2c-7c59-4d3b-bce8-0b2ad7aeaf09-kube-api-access-77x4k\") pod \"kube-auth-proxy-7f5c59f4bf-xkxqh\" (UID: \"28e63d2c-7c59-4d3b-bce8-0b2ad7aeaf09\") " pod="openshift-ingress/kube-auth-proxy-7f5c59f4bf-xkxqh" Jul 23 04:46:21.037214 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:46:21.037160 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28e63d2c-7c59-4d3b-bce8-0b2ad7aeaf09-tls-certs\") pod \"kube-auth-proxy-7f5c59f4bf-xkxqh\" (UID: \"28e63d2c-7c59-4d3b-bce8-0b2ad7aeaf09\") " pod="openshift-ingress/kube-auth-proxy-7f5c59f4bf-xkxqh" Jul 23 04:46:21.037394 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:46:21.037237 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/28e63d2c-7c59-4d3b-bce8-0b2ad7aeaf09-tmp\") pod \"kube-auth-proxy-7f5c59f4bf-xkxqh\" (UID: \"28e63d2c-7c59-4d3b-bce8-0b2ad7aeaf09\") " pod="openshift-ingress/kube-auth-proxy-7f5c59f4bf-xkxqh" Jul 23 04:46:21.037394 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:46:21.037279 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77x4k\" (UniqueName: \"kubernetes.io/projected/28e63d2c-7c59-4d3b-bce8-0b2ad7aeaf09-kube-api-access-77x4k\") pod \"kube-auth-proxy-7f5c59f4bf-xkxqh\" (UID: \"28e63d2c-7c59-4d3b-bce8-0b2ad7aeaf09\") " pod="openshift-ingress/kube-auth-proxy-7f5c59f4bf-xkxqh" Jul 23 04:46:21.039562 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:46:21.039533 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/28e63d2c-7c59-4d3b-bce8-0b2ad7aeaf09-tmp\") pod \"kube-auth-proxy-7f5c59f4bf-xkxqh\" (UID: \"28e63d2c-7c59-4d3b-bce8-0b2ad7aeaf09\") " pod="openshift-ingress/kube-auth-proxy-7f5c59f4bf-xkxqh" Jul 23 04:46:21.039666 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:46:21.039648 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28e63d2c-7c59-4d3b-bce8-0b2ad7aeaf09-tls-certs\") pod \"kube-auth-proxy-7f5c59f4bf-xkxqh\" (UID: \"28e63d2c-7c59-4d3b-bce8-0b2ad7aeaf09\") " pod="openshift-ingress/kube-auth-proxy-7f5c59f4bf-xkxqh" Jul 23 04:46:21.048055 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:46:21.048030 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77x4k\" (UniqueName: \"kubernetes.io/projected/28e63d2c-7c59-4d3b-bce8-0b2ad7aeaf09-kube-api-access-77x4k\") pod \"kube-auth-proxy-7f5c59f4bf-xkxqh\" (UID: \"28e63d2c-7c59-4d3b-bce8-0b2ad7aeaf09\") " pod="openshift-ingress/kube-auth-proxy-7f5c59f4bf-xkxqh" Jul 23 04:46:21.123598 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:46:21.123564 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7f5c59f4bf-xkxqh" Jul 23 04:46:21.261110 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:46:21.261083 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7f5c59f4bf-xkxqh"] Jul 23 04:46:21.263128 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:46:21.263101 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28e63d2c_7c59_4d3b_bce8_0b2ad7aeaf09.slice/crio-c646599ac8c3602ff9ab125659ee0c0e22a36668c5aece213a6795b9a069a3ec WatchSource:0}: Error finding container c646599ac8c3602ff9ab125659ee0c0e22a36668c5aece213a6795b9a069a3ec: Status 404 returned error can't find the container with id c646599ac8c3602ff9ab125659ee0c0e22a36668c5aece213a6795b9a069a3ec Jul 23 04:46:22.083978 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:46:22.083937 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7f5c59f4bf-xkxqh" event={"ID":"28e63d2c-7c59-4d3b-bce8-0b2ad7aeaf09","Type":"ContainerStarted","Data":"c646599ac8c3602ff9ab125659ee0c0e22a36668c5aece213a6795b9a069a3ec"} Jul 23 04:46:24.096147 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:46:24.096101 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7f5c59f4bf-xkxqh" event={"ID":"28e63d2c-7c59-4d3b-bce8-0b2ad7aeaf09","Type":"ContainerStarted","Data":"f6b4174be84a8bc9f82c68a99e200a9137710efcca58be34a0346ebd1dbe5aa0"} Jul 23 04:46:24.114314 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:46:24.114262 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7f5c59f4bf-xkxqh" podStartSLOduration=1.8304763259999999 podStartE2EDuration="4.114247545s" totalImagesPullingTime="2.283771219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:46:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:46:24.112910881 +0000 UTC m=+419.455423767" watchObservedRunningTime="2026-07-23 04:46:24.114247545 +0000 UTC m=+419.456760423" Jul 23 04:47:57.599595 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:47:57.599562 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-8srbc"] Jul 23 04:47:57.602959 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:47:57.602944 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8srbc" Jul 23 04:47:57.610541 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:47:57.610509 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 23 04:47:57.610666 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:47:57.610599 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 23 04:47:57.610666 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:47:57.610635 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 23 04:47:57.613661 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:47:57.613646 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 23 04:47:57.613873 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:47:57.613859 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-27kbg\"" Jul 23 04:47:57.625765 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:47:57.625743 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-8srbc"] Jul 23 04:47:57.667342 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:47:57.667317 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/def40d9d-1587-4886-b14e-7c114f255e6e-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-8srbc\" (UID: \"def40d9d-1587-4886-b14e-7c114f255e6e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8srbc" Jul 23 04:47:57.667450 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:47:57.667354 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/def40d9d-1587-4886-b14e-7c114f255e6e-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-8srbc\" (UID: \"def40d9d-1587-4886-b14e-7c114f255e6e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8srbc" Jul 23 04:47:57.667450 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:47:57.667391 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mz2cx\" (UniqueName: \"kubernetes.io/projected/def40d9d-1587-4886-b14e-7c114f255e6e-kube-api-access-mz2cx\") pod \"kuadrant-console-plugin-66b5774985-8srbc\" (UID: \"def40d9d-1587-4886-b14e-7c114f255e6e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8srbc" Jul 23 04:47:57.767917 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:47:57.767888 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mz2cx\" (UniqueName: \"kubernetes.io/projected/def40d9d-1587-4886-b14e-7c114f255e6e-kube-api-access-mz2cx\") pod \"kuadrant-console-plugin-66b5774985-8srbc\" (UID: \"def40d9d-1587-4886-b14e-7c114f255e6e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8srbc" Jul 23 04:47:57.768106 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:47:57.767947 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/def40d9d-1587-4886-b14e-7c114f255e6e-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-8srbc\" (UID: \"def40d9d-1587-4886-b14e-7c114f255e6e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8srbc" Jul 23 04:47:57.768106 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:47:57.767972 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/def40d9d-1587-4886-b14e-7c114f255e6e-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-8srbc\" (UID: \"def40d9d-1587-4886-b14e-7c114f255e6e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8srbc" Jul 23 04:47:57.768514 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:47:57.768497 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/def40d9d-1587-4886-b14e-7c114f255e6e-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-8srbc\" (UID: \"def40d9d-1587-4886-b14e-7c114f255e6e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8srbc" Jul 23 04:47:57.770649 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:47:57.770621 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/def40d9d-1587-4886-b14e-7c114f255e6e-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-8srbc\" (UID: \"def40d9d-1587-4886-b14e-7c114f255e6e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8srbc" Jul 23 04:47:57.783536 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:47:57.783507 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mz2cx\" (UniqueName: \"kubernetes.io/projected/def40d9d-1587-4886-b14e-7c114f255e6e-kube-api-access-mz2cx\") pod \"kuadrant-console-plugin-66b5774985-8srbc\" (UID: \"def40d9d-1587-4886-b14e-7c114f255e6e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8srbc" Jul 23 04:47:57.925022 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:47:57.924949 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8srbc" Jul 23 04:47:58.058165 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:47:58.058132 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-8srbc"] Jul 23 04:47:58.061171 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:47:58.061141 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddef40d9d_1587_4886_b14e_7c114f255e6e.slice/crio-f006d105344734e32c1ddfd57ada8e76c024c6f75fc51c82a804ca05a978f780 WatchSource:0}: Error finding container f006d105344734e32c1ddfd57ada8e76c024c6f75fc51c82a804ca05a978f780: Status 404 returned error can't find the container with id f006d105344734e32c1ddfd57ada8e76c024c6f75fc51c82a804ca05a978f780 Jul 23 04:47:58.511111 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:47:58.511080 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8srbc" event={"ID":"def40d9d-1587-4886-b14e-7c114f255e6e","Type":"ContainerStarted","Data":"f006d105344734e32c1ddfd57ada8e76c024c6f75fc51c82a804ca05a978f780"} Jul 23 04:48:20.628393 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:48:20.628352 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8srbc" event={"ID":"def40d9d-1587-4886-b14e-7c114f255e6e","Type":"ContainerStarted","Data":"568ee300d378a681baf2f293f43520dbc9798d0c952c3cc50dd63e42b5dc21ca"} Jul 23 04:48:20.682545 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:48:20.682486 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8srbc" podStartSLOduration=1.616522111 podStartE2EDuration="23.682467043s" totalImagesPullingTime="22.065944932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:47:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:48:20.678261832 +0000 UTC m=+536.020774720" watchObservedRunningTime="2026-07-23 04:48:20.682467043 +0000 UTC m=+536.024979931" Jul 23 04:48:43.326347 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:48:43.326309 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 04:48:43.337866 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:48:43.337823 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-nvp4j" Jul 23 04:48:43.340470 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:48:43.340442 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 23 04:48:43.343346 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:48:43.343325 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 04:48:43.357461 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:48:43.357434 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 04:48:43.398453 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:48:43.398424 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4c4a8f86-e655-46fe-9371-f9a6f10501dc-config-file\") pod \"limitador-limitador-54d6d988d5-nvp4j\" (UID: \"4c4a8f86-e655-46fe-9371-f9a6f10501dc\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nvp4j" Jul 23 04:48:43.398627 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:48:43.398466 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql5dz\" (UniqueName: \"kubernetes.io/projected/4c4a8f86-e655-46fe-9371-f9a6f10501dc-kube-api-access-ql5dz\") pod \"limitador-limitador-54d6d988d5-nvp4j\" (UID: \"4c4a8f86-e655-46fe-9371-f9a6f10501dc\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nvp4j" Jul 23 04:48:43.499180 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:48:43.499147 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql5dz\" (UniqueName: \"kubernetes.io/projected/4c4a8f86-e655-46fe-9371-f9a6f10501dc-kube-api-access-ql5dz\") pod \"limitador-limitador-54d6d988d5-nvp4j\" (UID: \"4c4a8f86-e655-46fe-9371-f9a6f10501dc\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nvp4j" Jul 23 04:48:43.499353 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:48:43.499325 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4c4a8f86-e655-46fe-9371-f9a6f10501dc-config-file\") pod \"limitador-limitador-54d6d988d5-nvp4j\" (UID: \"4c4a8f86-e655-46fe-9371-f9a6f10501dc\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nvp4j" Jul 23 04:48:43.499976 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:48:43.499958 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4c4a8f86-e655-46fe-9371-f9a6f10501dc-config-file\") pod \"limitador-limitador-54d6d988d5-nvp4j\" (UID: \"4c4a8f86-e655-46fe-9371-f9a6f10501dc\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nvp4j" Jul 23 04:48:43.508731 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:48:43.508699 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql5dz\" (UniqueName: \"kubernetes.io/projected/4c4a8f86-e655-46fe-9371-f9a6f10501dc-kube-api-access-ql5dz\") pod \"limitador-limitador-54d6d988d5-nvp4j\" (UID: \"4c4a8f86-e655-46fe-9371-f9a6f10501dc\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nvp4j" Jul 23 04:48:43.649086 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:48:43.649008 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-nvp4j" Jul 23 04:48:43.895229 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:48:43.895204 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 04:48:43.897367 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:48:43.897342 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c4a8f86_e655_46fe_9371_f9a6f10501dc.slice/crio-b921e9355dd4d61a035d7ef7d5b9fead217fc2bf6959b19e61955c8bd84ba8f1 WatchSource:0}: Error finding container b921e9355dd4d61a035d7ef7d5b9fead217fc2bf6959b19e61955c8bd84ba8f1: Status 404 returned error can't find the container with id b921e9355dd4d61a035d7ef7d5b9fead217fc2bf6959b19e61955c8bd84ba8f1 Jul 23 04:48:44.167366 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:48:44.167285 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-jqhn9"] Jul 23 04:48:44.185349 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:48:44.185318 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-jqhn9"] Jul 23 04:48:44.185499 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:48:44.185405 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-jqhn9" Jul 23 04:48:44.188331 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:48:44.188312 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-7929q\"" Jul 23 04:48:44.306979 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:48:44.306944 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xd7gn\" (UniqueName: \"kubernetes.io/projected/c0e71633-7df7-417f-9dc7-d5bf3fc38412-kube-api-access-xd7gn\") pod \"authorino-75d556bd77-jqhn9\" (UID: \"c0e71633-7df7-417f-9dc7-d5bf3fc38412\") " pod="kuadrant-system/authorino-75d556bd77-jqhn9" Jul 23 04:48:44.407544 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:48:44.407505 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd7gn\" (UniqueName: \"kubernetes.io/projected/c0e71633-7df7-417f-9dc7-d5bf3fc38412-kube-api-access-xd7gn\") pod \"authorino-75d556bd77-jqhn9\" (UID: \"c0e71633-7df7-417f-9dc7-d5bf3fc38412\") " pod="kuadrant-system/authorino-75d556bd77-jqhn9" Jul 23 04:48:44.431483 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:48:44.431374 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xd7gn\" (UniqueName: \"kubernetes.io/projected/c0e71633-7df7-417f-9dc7-d5bf3fc38412-kube-api-access-xd7gn\") pod \"authorino-75d556bd77-jqhn9\" (UID: \"c0e71633-7df7-417f-9dc7-d5bf3fc38412\") " pod="kuadrant-system/authorino-75d556bd77-jqhn9" Jul 23 04:48:44.495578 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:48:44.495516 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-jqhn9" Jul 23 04:48:44.700068 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:48:44.700040 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-jqhn9"] Jul 23 04:48:44.702400 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:48:44.702369 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc0e71633_7df7_417f_9dc7_d5bf3fc38412.slice/crio-7380d50c1d0d9e9e6e61efef42804d6a458410f615a99045e06cab50f6b5d069 WatchSource:0}: Error finding container 7380d50c1d0d9e9e6e61efef42804d6a458410f615a99045e06cab50f6b5d069: Status 404 returned error can't find the container with id 7380d50c1d0d9e9e6e61efef42804d6a458410f615a99045e06cab50f6b5d069 Jul 23 04:48:44.738455 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:48:44.738422 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-nvp4j" event={"ID":"4c4a8f86-e655-46fe-9371-f9a6f10501dc","Type":"ContainerStarted","Data":"b921e9355dd4d61a035d7ef7d5b9fead217fc2bf6959b19e61955c8bd84ba8f1"} Jul 23 04:48:44.739923 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:48:44.739889 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-jqhn9" event={"ID":"c0e71633-7df7-417f-9dc7-d5bf3fc38412","Type":"ContainerStarted","Data":"7380d50c1d0d9e9e6e61efef42804d6a458410f615a99045e06cab50f6b5d069"} Jul 23 04:48:46.753741 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:48:46.753628 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-nvp4j" event={"ID":"4c4a8f86-e655-46fe-9371-f9a6f10501dc","Type":"ContainerStarted","Data":"3a01232b80dfafc4693c1bc6e99cdb1e4b20fcbe5f60ed547627eb5f6d65741f"} Jul 23 04:48:46.754226 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:48:46.753872 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-nvp4j" Jul 23 04:48:46.775711 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:48:46.775598 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-nvp4j" podStartSLOduration=1.1658475369999999 podStartE2EDuration="3.775579664s" totalImagesPullingTime="2.609732127s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:48:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:48:46.775175948 +0000 UTC m=+562.117688838" watchObservedRunningTime="2026-07-23 04:48:46.775579664 +0000 UTC m=+562.118092551" Jul 23 04:48:48.764707 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:48:48.764671 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-jqhn9" event={"ID":"c0e71633-7df7-417f-9dc7-d5bf3fc38412","Type":"ContainerStarted","Data":"1a156a607dadac8655068fd00c4b515b88fd8f18b2e360c995ed2e6cade92a93"} Jul 23 04:48:48.788576 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:48:48.788529 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-jqhn9" podStartSLOduration=0.946034058 podStartE2EDuration="4.78851435s" totalImagesPullingTime="3.842480292s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:48:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:48:48.787149857 +0000 UTC m=+564.129662746" watchObservedRunningTime="2026-07-23 04:48:48.78851435 +0000 UTC m=+564.131027280" Jul 23 04:48:57.760108 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:48:57.760079 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-nvp4j" Jul 23 04:49:14.152653 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:49:14.147637 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-jqhn9"] Jul 23 04:49:14.152653 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:49:14.147897 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-jqhn9" podUID="c0e71633-7df7-417f-9dc7-d5bf3fc38412" containerName="authorino" containerID="cri-o://1a156a607dadac8655068fd00c4b515b88fd8f18b2e360c995ed2e6cade92a93" gracePeriod=30 Jul 23 04:49:14.399949 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:49:14.399920 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-jqhn9" Jul 23 04:49:14.504618 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:49:14.504518 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xd7gn\" (UniqueName: \"kubernetes.io/projected/c0e71633-7df7-417f-9dc7-d5bf3fc38412-kube-api-access-xd7gn\") pod \"c0e71633-7df7-417f-9dc7-d5bf3fc38412\" (UID: \"c0e71633-7df7-417f-9dc7-d5bf3fc38412\") " Jul 23 04:49:14.506770 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:49:14.506741 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0e71633-7df7-417f-9dc7-d5bf3fc38412-kube-api-access-xd7gn" (OuterVolumeSpecName: "kube-api-access-xd7gn") pod "c0e71633-7df7-417f-9dc7-d5bf3fc38412" (UID: "c0e71633-7df7-417f-9dc7-d5bf3fc38412"). InnerVolumeSpecName "kube-api-access-xd7gn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 04:49:14.606009 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:49:14.605978 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xd7gn\" (UniqueName: \"kubernetes.io/projected/c0e71633-7df7-417f-9dc7-d5bf3fc38412-kube-api-access-xd7gn\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 04:49:14.883537 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:49:14.883496 2562 generic.go:356] "Generic (PLEG): container finished" podID="c0e71633-7df7-417f-9dc7-d5bf3fc38412" containerID="1a156a607dadac8655068fd00c4b515b88fd8f18b2e360c995ed2e6cade92a93" exitCode=0 Jul 23 04:49:14.883735 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:49:14.883550 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-jqhn9" Jul 23 04:49:14.883735 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:49:14.883608 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-jqhn9" event={"ID":"c0e71633-7df7-417f-9dc7-d5bf3fc38412","Type":"ContainerDied","Data":"1a156a607dadac8655068fd00c4b515b88fd8f18b2e360c995ed2e6cade92a93"} Jul 23 04:49:14.883735 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:49:14.883658 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-jqhn9" event={"ID":"c0e71633-7df7-417f-9dc7-d5bf3fc38412","Type":"ContainerDied","Data":"7380d50c1d0d9e9e6e61efef42804d6a458410f615a99045e06cab50f6b5d069"} Jul 23 04:49:14.883735 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:49:14.883681 2562 scope.go:117] "RemoveContainer" containerID="1a156a607dadac8655068fd00c4b515b88fd8f18b2e360c995ed2e6cade92a93" Jul 23 04:49:14.892487 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:49:14.892471 2562 scope.go:117] "RemoveContainer" containerID="1a156a607dadac8655068fd00c4b515b88fd8f18b2e360c995ed2e6cade92a93" Jul 23 04:49:14.892759 ip-10-0-128-96 kubenswrapper[2562]: E0723 04:49:14.892742 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a156a607dadac8655068fd00c4b515b88fd8f18b2e360c995ed2e6cade92a93\": container with ID starting with 1a156a607dadac8655068fd00c4b515b88fd8f18b2e360c995ed2e6cade92a93 not found: ID does not exist" containerID="1a156a607dadac8655068fd00c4b515b88fd8f18b2e360c995ed2e6cade92a93" Jul 23 04:49:14.892814 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:49:14.892766 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a156a607dadac8655068fd00c4b515b88fd8f18b2e360c995ed2e6cade92a93"} err="failed to get container status \"1a156a607dadac8655068fd00c4b515b88fd8f18b2e360c995ed2e6cade92a93\": rpc error: code = NotFound desc = could not find container \"1a156a607dadac8655068fd00c4b515b88fd8f18b2e360c995ed2e6cade92a93\": container with ID starting with 1a156a607dadac8655068fd00c4b515b88fd8f18b2e360c995ed2e6cade92a93 not found: ID does not exist" Jul 23 04:49:14.908584 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:49:14.908559 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-jqhn9"] Jul 23 04:49:14.916825 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:49:14.916804 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-jqhn9"] Jul 23 04:49:15.206873 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:49:15.206767 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0e71633-7df7-417f-9dc7-d5bf3fc38412" path="/var/lib/kubelet/pods/c0e71633-7df7-417f-9dc7-d5bf3fc38412/volumes" Jul 23 04:49:25.101714 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:49:25.101687 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vphbq_b3007292-2534-4dc0-82a3-41d6a7d34c0c/ovn-acl-logging/0.log" Jul 23 04:49:25.105603 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:49:25.105583 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vphbq_b3007292-2534-4dc0-82a3-41d6a7d34c0c/ovn-acl-logging/0.log" Jul 23 04:49:55.956701 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:49:55.956627 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 04:50:11.905277 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:11.905245 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 04:50:25.883395 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:25.883357 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 04:50:34.996297 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:34.996254 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 04:50:38.549580 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:38.549544 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg"] Jul 23 04:50:38.549960 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:38.549894 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0e71633-7df7-417f-9dc7-d5bf3fc38412" containerName="authorino" Jul 23 04:50:38.549960 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:38.549905 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0e71633-7df7-417f-9dc7-d5bf3fc38412" containerName="authorino" Jul 23 04:50:38.550064 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:38.549970 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0e71633-7df7-417f-9dc7-d5bf3fc38412" containerName="authorino" Jul 23 04:50:38.551960 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:38.551945 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg" Jul 23 04:50:38.555089 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:38.555069 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 23 04:50:38.555750 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:38.555733 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 23 04:50:38.556183 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:38.556165 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-f7qjs\"" Jul 23 04:50:38.556259 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:38.556205 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 23 04:50:38.565080 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:38.565059 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg"] Jul 23 04:50:38.647813 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:38.647769 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b48eb5e-522f-4e02-9bb0-902f172b3a06-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg\" (UID: \"4b48eb5e-522f-4e02-9bb0-902f172b3a06\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg" Jul 23 04:50:38.648015 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:38.647889 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b48eb5e-522f-4e02-9bb0-902f172b3a06-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg\" (UID: \"4b48eb5e-522f-4e02-9bb0-902f172b3a06\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg" Jul 23 04:50:38.648015 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:38.647941 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b48eb5e-522f-4e02-9bb0-902f172b3a06-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg\" (UID: \"4b48eb5e-522f-4e02-9bb0-902f172b3a06\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg" Jul 23 04:50:38.648015 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:38.647988 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b48eb5e-522f-4e02-9bb0-902f172b3a06-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg\" (UID: \"4b48eb5e-522f-4e02-9bb0-902f172b3a06\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg" Jul 23 04:50:38.648015 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:38.648011 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b48eb5e-522f-4e02-9bb0-902f172b3a06-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg\" (UID: \"4b48eb5e-522f-4e02-9bb0-902f172b3a06\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg" Jul 23 04:50:38.648196 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:38.648035 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlwsj\" (UniqueName: \"kubernetes.io/projected/4b48eb5e-522f-4e02-9bb0-902f172b3a06-kube-api-access-nlwsj\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg\" (UID: \"4b48eb5e-522f-4e02-9bb0-902f172b3a06\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg" Jul 23 04:50:38.749200 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:38.749146 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b48eb5e-522f-4e02-9bb0-902f172b3a06-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg\" (UID: \"4b48eb5e-522f-4e02-9bb0-902f172b3a06\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg" Jul 23 04:50:38.749398 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:38.749238 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b48eb5e-522f-4e02-9bb0-902f172b3a06-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg\" (UID: \"4b48eb5e-522f-4e02-9bb0-902f172b3a06\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg" Jul 23 04:50:38.749398 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:38.749279 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b48eb5e-522f-4e02-9bb0-902f172b3a06-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg\" (UID: \"4b48eb5e-522f-4e02-9bb0-902f172b3a06\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg" Jul 23 04:50:38.749398 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:38.749314 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b48eb5e-522f-4e02-9bb0-902f172b3a06-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg\" (UID: \"4b48eb5e-522f-4e02-9bb0-902f172b3a06\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg" Jul 23 04:50:38.749398 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:38.749337 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b48eb5e-522f-4e02-9bb0-902f172b3a06-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg\" (UID: \"4b48eb5e-522f-4e02-9bb0-902f172b3a06\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg" Jul 23 04:50:38.749398 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:38.749364 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlwsj\" (UniqueName: \"kubernetes.io/projected/4b48eb5e-522f-4e02-9bb0-902f172b3a06-kube-api-access-nlwsj\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg\" (UID: \"4b48eb5e-522f-4e02-9bb0-902f172b3a06\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg" Jul 23 04:50:38.749713 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:38.749689 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b48eb5e-522f-4e02-9bb0-902f172b3a06-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg\" (UID: \"4b48eb5e-522f-4e02-9bb0-902f172b3a06\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg" Jul 23 04:50:38.749771 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:38.749746 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b48eb5e-522f-4e02-9bb0-902f172b3a06-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg\" (UID: \"4b48eb5e-522f-4e02-9bb0-902f172b3a06\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg" Jul 23 04:50:38.749811 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:38.749756 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b48eb5e-522f-4e02-9bb0-902f172b3a06-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg\" (UID: \"4b48eb5e-522f-4e02-9bb0-902f172b3a06\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg" Jul 23 04:50:38.751639 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:38.751617 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b48eb5e-522f-4e02-9bb0-902f172b3a06-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg\" (UID: \"4b48eb5e-522f-4e02-9bb0-902f172b3a06\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg" Jul 23 04:50:38.751745 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:38.751726 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b48eb5e-522f-4e02-9bb0-902f172b3a06-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg\" (UID: \"4b48eb5e-522f-4e02-9bb0-902f172b3a06\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg" Jul 23 04:50:38.759766 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:38.759745 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlwsj\" (UniqueName: \"kubernetes.io/projected/4b48eb5e-522f-4e02-9bb0-902f172b3a06-kube-api-access-nlwsj\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg\" (UID: \"4b48eb5e-522f-4e02-9bb0-902f172b3a06\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg" Jul 23 04:50:38.862649 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:38.862610 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg" Jul 23 04:50:39.003855 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:39.003798 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg"] Jul 23 04:50:39.007662 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:50:39.007638 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b48eb5e_522f_4e02_9bb0_902f172b3a06.slice/crio-c0e4e41479d797e10db1bf609e5cf4babaca525080d5c21b232d90cf2d03eff9 WatchSource:0}: Error finding container c0e4e41479d797e10db1bf609e5cf4babaca525080d5c21b232d90cf2d03eff9: Status 404 returned error can't find the container with id c0e4e41479d797e10db1bf609e5cf4babaca525080d5c21b232d90cf2d03eff9 Jul 23 04:50:39.261430 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:39.261273 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg" event={"ID":"4b48eb5e-522f-4e02-9bb0-902f172b3a06","Type":"ContainerStarted","Data":"c0e4e41479d797e10db1bf609e5cf4babaca525080d5c21b232d90cf2d03eff9"} Jul 23 04:50:39.984245 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:39.984209 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 04:50:40.267267 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:40.267175 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg" event={"ID":"4b48eb5e-522f-4e02-9bb0-902f172b3a06","Type":"ContainerStarted","Data":"c7d4911a46ae7903d20a3b28757c9c1bb9af25aebf669c1419824d4212a843e9"} Jul 23 04:50:40.288238 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:40.288186 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg" podStartSLOduration=1.3069098829999999 podStartE2EDuration="2.288170853s" totalImagesPullingTime="981.26097ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:50:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:50:40.287425113 +0000 UTC m=+675.629938003" watchObservedRunningTime="2026-07-23 04:50:40.288170853 +0000 UTC m=+675.630683774" Jul 23 04:50:48.863795 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:48.863759 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg" Jul 23 04:50:48.864214 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:48.863823 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg" Jul 23 04:50:48.876270 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:48.876243 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg" Jul 23 04:50:49.319541 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:49.319458 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg" Jul 23 04:50:50.904377 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:50.904344 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-8vgln"] Jul 23 04:50:50.906552 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:50.906536 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8vgln" Jul 23 04:50:50.910454 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:50.910430 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 23 04:50:50.921925 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:50.921896 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-8vgln"] Jul 23 04:50:50.965390 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:50.965352 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe4557e0-dad6-477e-b318-81579698c848-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8vgln\" (UID: \"fe4557e0-dad6-477e-b318-81579698c848\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8vgln" Jul 23 04:50:50.965602 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:50.965408 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vqj9\" (UniqueName: \"kubernetes.io/projected/fe4557e0-dad6-477e-b318-81579698c848-kube-api-access-2vqj9\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8vgln\" (UID: \"fe4557e0-dad6-477e-b318-81579698c848\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8vgln" Jul 23 04:50:50.965602 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:50.965475 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe4557e0-dad6-477e-b318-81579698c848-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8vgln\" (UID: \"fe4557e0-dad6-477e-b318-81579698c848\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8vgln" Jul 23 04:50:50.965602 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:50.965564 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe4557e0-dad6-477e-b318-81579698c848-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8vgln\" (UID: \"fe4557e0-dad6-477e-b318-81579698c848\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8vgln" Jul 23 04:50:50.965602 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:50.965590 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe4557e0-dad6-477e-b318-81579698c848-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8vgln\" (UID: \"fe4557e0-dad6-477e-b318-81579698c848\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8vgln" Jul 23 04:50:50.965773 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:50.965612 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe4557e0-dad6-477e-b318-81579698c848-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8vgln\" (UID: \"fe4557e0-dad6-477e-b318-81579698c848\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8vgln" Jul 23 04:50:51.066719 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:51.066677 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vqj9\" (UniqueName: \"kubernetes.io/projected/fe4557e0-dad6-477e-b318-81579698c848-kube-api-access-2vqj9\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8vgln\" (UID: \"fe4557e0-dad6-477e-b318-81579698c848\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8vgln" Jul 23 04:50:51.066953 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:51.066728 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe4557e0-dad6-477e-b318-81579698c848-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8vgln\" (UID: \"fe4557e0-dad6-477e-b318-81579698c848\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8vgln" Jul 23 04:50:51.066953 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:51.066781 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe4557e0-dad6-477e-b318-81579698c848-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8vgln\" (UID: \"fe4557e0-dad6-477e-b318-81579698c848\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8vgln" Jul 23 04:50:51.066953 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:51.066799 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe4557e0-dad6-477e-b318-81579698c848-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8vgln\" (UID: \"fe4557e0-dad6-477e-b318-81579698c848\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8vgln" Jul 23 04:50:51.066953 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:51.066818 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe4557e0-dad6-477e-b318-81579698c848-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8vgln\" (UID: \"fe4557e0-dad6-477e-b318-81579698c848\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8vgln" Jul 23 04:50:51.067153 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:51.066978 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe4557e0-dad6-477e-b318-81579698c848-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8vgln\" (UID: \"fe4557e0-dad6-477e-b318-81579698c848\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8vgln" Jul 23 04:50:51.067325 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:51.067288 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe4557e0-dad6-477e-b318-81579698c848-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8vgln\" (UID: \"fe4557e0-dad6-477e-b318-81579698c848\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8vgln" Jul 23 04:50:51.067325 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:51.067308 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe4557e0-dad6-477e-b318-81579698c848-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8vgln\" (UID: \"fe4557e0-dad6-477e-b318-81579698c848\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8vgln" Jul 23 04:50:51.067495 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:51.067341 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe4557e0-dad6-477e-b318-81579698c848-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8vgln\" (UID: \"fe4557e0-dad6-477e-b318-81579698c848\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8vgln" Jul 23 04:50:51.069178 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:51.069152 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe4557e0-dad6-477e-b318-81579698c848-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8vgln\" (UID: \"fe4557e0-dad6-477e-b318-81579698c848\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8vgln" Jul 23 04:50:51.069326 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:51.069312 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe4557e0-dad6-477e-b318-81579698c848-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8vgln\" (UID: \"fe4557e0-dad6-477e-b318-81579698c848\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8vgln" Jul 23 04:50:51.079080 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:51.079054 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vqj9\" (UniqueName: \"kubernetes.io/projected/fe4557e0-dad6-477e-b318-81579698c848-kube-api-access-2vqj9\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8vgln\" (UID: \"fe4557e0-dad6-477e-b318-81579698c848\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8vgln" Jul 23 04:50:51.216583 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:51.216498 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8vgln" Jul 23 04:50:51.359079 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:51.359043 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-8vgln"] Jul 23 04:50:51.362257 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:50:51.362231 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe4557e0_dad6_477e_b318_81579698c848.slice/crio-714ddd6b9b16df409bebcee66700b83a1d243a45b988736c2fc9c7e91da4ed12 WatchSource:0}: Error finding container 714ddd6b9b16df409bebcee66700b83a1d243a45b988736c2fc9c7e91da4ed12: Status 404 returned error can't find the container with id 714ddd6b9b16df409bebcee66700b83a1d243a45b988736c2fc9c7e91da4ed12 Jul 23 04:50:51.363563 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:51.363548 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 04:50:52.322519 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:52.322484 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8vgln" event={"ID":"fe4557e0-dad6-477e-b318-81579698c848","Type":"ContainerStarted","Data":"332c156b66dc8cb2331d06588a3f640deb0ca99d30799fb9949d6a8299286429"} Jul 23 04:50:52.322944 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:52.322566 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8vgln" event={"ID":"fe4557e0-dad6-477e-b318-81579698c848","Type":"ContainerStarted","Data":"714ddd6b9b16df409bebcee66700b83a1d243a45b988736c2fc9c7e91da4ed12"} Jul 23 04:50:53.828717 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:53.828658 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8vgln" podStartSLOduration=3.647993039 podStartE2EDuration="3.828640849s" totalImagesPullingTime="180.64781ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:50:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:50:52.349015766 +0000 UTC m=+687.691528655" watchObservedRunningTime="2026-07-23 04:50:53.828640849 +0000 UTC m=+689.171153717" Jul 23 04:50:53.829117 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:53.828802 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 04:50:59.661225 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:59.661186 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-thp2s"] Jul 23 04:50:59.663981 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:59.663961 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thp2s" Jul 23 04:50:59.666747 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:59.666721 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 23 04:50:59.674659 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:59.674632 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-thp2s"] Jul 23 04:50:59.751322 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:59.751287 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5777220-879c-4109-af66-cb99d923565e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-thp2s\" (UID: \"d5777220-879c-4109-af66-cb99d923565e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thp2s" Jul 23 04:50:59.751322 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:59.751328 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlhc9\" (UniqueName: \"kubernetes.io/projected/d5777220-879c-4109-af66-cb99d923565e-kube-api-access-dlhc9\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-thp2s\" (UID: \"d5777220-879c-4109-af66-cb99d923565e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thp2s" Jul 23 04:50:59.751525 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:59.751360 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5777220-879c-4109-af66-cb99d923565e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-thp2s\" (UID: \"d5777220-879c-4109-af66-cb99d923565e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thp2s" Jul 23 04:50:59.751525 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:59.751431 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5777220-879c-4109-af66-cb99d923565e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-thp2s\" (UID: \"d5777220-879c-4109-af66-cb99d923565e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thp2s" Jul 23 04:50:59.751525 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:59.751456 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5777220-879c-4109-af66-cb99d923565e-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-thp2s\" (UID: \"d5777220-879c-4109-af66-cb99d923565e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thp2s" Jul 23 04:50:59.751525 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:59.751501 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5777220-879c-4109-af66-cb99d923565e-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-thp2s\" (UID: \"d5777220-879c-4109-af66-cb99d923565e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thp2s" Jul 23 04:50:59.852212 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:59.852175 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5777220-879c-4109-af66-cb99d923565e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-thp2s\" (UID: \"d5777220-879c-4109-af66-cb99d923565e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thp2s" Jul 23 04:50:59.852212 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:59.852219 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlhc9\" (UniqueName: \"kubernetes.io/projected/d5777220-879c-4109-af66-cb99d923565e-kube-api-access-dlhc9\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-thp2s\" (UID: \"d5777220-879c-4109-af66-cb99d923565e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thp2s" Jul 23 04:50:59.852445 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:59.852250 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5777220-879c-4109-af66-cb99d923565e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-thp2s\" (UID: \"d5777220-879c-4109-af66-cb99d923565e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thp2s" Jul 23 04:50:59.852445 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:59.852288 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5777220-879c-4109-af66-cb99d923565e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-thp2s\" (UID: \"d5777220-879c-4109-af66-cb99d923565e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thp2s" Jul 23 04:50:59.852445 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:59.852314 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5777220-879c-4109-af66-cb99d923565e-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-thp2s\" (UID: \"d5777220-879c-4109-af66-cb99d923565e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thp2s" Jul 23 04:50:59.852445 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:59.852338 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5777220-879c-4109-af66-cb99d923565e-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-thp2s\" (UID: \"d5777220-879c-4109-af66-cb99d923565e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thp2s" Jul 23 04:50:59.852920 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:59.852656 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5777220-879c-4109-af66-cb99d923565e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-thp2s\" (UID: \"d5777220-879c-4109-af66-cb99d923565e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thp2s" Jul 23 04:50:59.852920 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:59.852663 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5777220-879c-4109-af66-cb99d923565e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-thp2s\" (UID: \"d5777220-879c-4109-af66-cb99d923565e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thp2s" Jul 23 04:50:59.852920 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:59.852759 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5777220-879c-4109-af66-cb99d923565e-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-thp2s\" (UID: \"d5777220-879c-4109-af66-cb99d923565e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thp2s" Jul 23 04:50:59.854662 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:59.854637 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5777220-879c-4109-af66-cb99d923565e-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-thp2s\" (UID: \"d5777220-879c-4109-af66-cb99d923565e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thp2s" Jul 23 04:50:59.854895 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:59.854877 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5777220-879c-4109-af66-cb99d923565e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-thp2s\" (UID: \"d5777220-879c-4109-af66-cb99d923565e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thp2s" Jul 23 04:50:59.861015 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:59.860995 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlhc9\" (UniqueName: \"kubernetes.io/projected/d5777220-879c-4109-af66-cb99d923565e-kube-api-access-dlhc9\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-thp2s\" (UID: \"d5777220-879c-4109-af66-cb99d923565e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thp2s" Jul 23 04:50:59.974134 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:50:59.974043 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thp2s" Jul 23 04:51:00.113387 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:51:00.113362 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-thp2s"] Jul 23 04:51:00.115617 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:51:00.115587 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd5777220_879c_4109_af66_cb99d923565e.slice/crio-905f4b8af34ef17fa7084160ba5f2855a44e218dc33d50054c6354ee05ed9990 WatchSource:0}: Error finding container 905f4b8af34ef17fa7084160ba5f2855a44e218dc33d50054c6354ee05ed9990: Status 404 returned error can't find the container with id 905f4b8af34ef17fa7084160ba5f2855a44e218dc33d50054c6354ee05ed9990 Jul 23 04:51:00.368283 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:51:00.368252 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thp2s" event={"ID":"d5777220-879c-4109-af66-cb99d923565e","Type":"ContainerStarted","Data":"90285a3b5d775df6a3e28d377ee28c8f80d740868c7bd29a07a6c0d8367059b0"} Jul 23 04:51:00.368461 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:51:00.368297 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thp2s" event={"ID":"d5777220-879c-4109-af66-cb99d923565e","Type":"ContainerStarted","Data":"905f4b8af34ef17fa7084160ba5f2855a44e218dc33d50054c6354ee05ed9990"} Jul 23 04:51:00.390289 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:51:00.390239 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thp2s" podStartSLOduration=1.224356142 podStartE2EDuration="1.39022314s" totalImagesPullingTime="165.866998ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:50:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:51:00.388460081 +0000 UTC m=+695.730972970" watchObservedRunningTime="2026-07-23 04:51:00.39022314 +0000 UTC m=+695.732736006" Jul 23 04:51:01.217598 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:51:01.217572 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8vgln" Jul 23 04:51:01.218018 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:51:01.217625 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8vgln" Jul 23 04:51:01.230726 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:51:01.230699 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8vgln" Jul 23 04:51:01.383892 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:51:01.383861 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8vgln" Jul 23 04:51:02.768527 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:51:02.768491 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 04:51:09.975010 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:51:09.974971 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thp2s" Jul 23 04:51:09.975010 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:51:09.975016 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thp2s" Jul 23 04:51:09.987280 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:51:09.987251 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thp2s" Jul 23 04:51:10.423786 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:51:10.423758 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thp2s" Jul 23 04:52:12.211366 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:52:12.211325 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-d6dd8599-flvk5"] Jul 23 04:52:12.214842 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:52:12.214811 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d6dd8599-flvk5" Jul 23 04:52:12.218254 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:52:12.218230 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 23 04:52:12.218384 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:52:12.218278 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-7929q\"" Jul 23 04:52:12.224220 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:52:12.224196 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-d6dd8599-flvk5"] Jul 23 04:52:12.313307 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:52:12.313261 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w985l\" (UniqueName: \"kubernetes.io/projected/97b0d574-ea8d-4bc0-a1f8-16cf95c45572-kube-api-access-w985l\") pod \"authorino-d6dd8599-flvk5\" (UID: \"97b0d574-ea8d-4bc0-a1f8-16cf95c45572\") " pod="kuadrant-system/authorino-d6dd8599-flvk5" Jul 23 04:52:12.313307 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:52:12.313305 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/97b0d574-ea8d-4bc0-a1f8-16cf95c45572-tls-cert\") pod \"authorino-d6dd8599-flvk5\" (UID: \"97b0d574-ea8d-4bc0-a1f8-16cf95c45572\") " pod="kuadrant-system/authorino-d6dd8599-flvk5" Jul 23 04:52:12.414488 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:52:12.414445 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w985l\" (UniqueName: \"kubernetes.io/projected/97b0d574-ea8d-4bc0-a1f8-16cf95c45572-kube-api-access-w985l\") pod \"authorino-d6dd8599-flvk5\" (UID: \"97b0d574-ea8d-4bc0-a1f8-16cf95c45572\") " pod="kuadrant-system/authorino-d6dd8599-flvk5" Jul 23 04:52:12.414488 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:52:12.414490 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/97b0d574-ea8d-4bc0-a1f8-16cf95c45572-tls-cert\") pod \"authorino-d6dd8599-flvk5\" (UID: \"97b0d574-ea8d-4bc0-a1f8-16cf95c45572\") " pod="kuadrant-system/authorino-d6dd8599-flvk5" Jul 23 04:52:12.417062 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:52:12.417042 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/97b0d574-ea8d-4bc0-a1f8-16cf95c45572-tls-cert\") pod \"authorino-d6dd8599-flvk5\" (UID: \"97b0d574-ea8d-4bc0-a1f8-16cf95c45572\") " pod="kuadrant-system/authorino-d6dd8599-flvk5" Jul 23 04:52:12.425034 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:52:12.425013 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w985l\" (UniqueName: \"kubernetes.io/projected/97b0d574-ea8d-4bc0-a1f8-16cf95c45572-kube-api-access-w985l\") pod \"authorino-d6dd8599-flvk5\" (UID: \"97b0d574-ea8d-4bc0-a1f8-16cf95c45572\") " pod="kuadrant-system/authorino-d6dd8599-flvk5" Jul 23 04:52:12.523972 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:52:12.523894 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d6dd8599-flvk5" Jul 23 04:52:12.663938 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:52:12.663911 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-d6dd8599-flvk5"] Jul 23 04:52:12.665562 ip-10-0-128-96 kubenswrapper[2562]: W0723 04:52:12.665528 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod97b0d574_ea8d_4bc0_a1f8_16cf95c45572.slice/crio-5eeff80db86faace02a9c7e07aaa55ac7ba78ef637083c449015029290cf3996 WatchSource:0}: Error finding container 5eeff80db86faace02a9c7e07aaa55ac7ba78ef637083c449015029290cf3996: Status 404 returned error can't find the container with id 5eeff80db86faace02a9c7e07aaa55ac7ba78ef637083c449015029290cf3996 Jul 23 04:52:12.717700 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:52:12.717664 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d6dd8599-flvk5" event={"ID":"97b0d574-ea8d-4bc0-a1f8-16cf95c45572","Type":"ContainerStarted","Data":"5eeff80db86faace02a9c7e07aaa55ac7ba78ef637083c449015029290cf3996"} Jul 23 04:52:13.724860 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:52:13.724790 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d6dd8599-flvk5" event={"ID":"97b0d574-ea8d-4bc0-a1f8-16cf95c45572","Type":"ContainerStarted","Data":"c502ff80da24935f44bee1765669b80c0c19c2b0d48f805af02d156a16143081"} Jul 23 04:52:13.745500 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:52:13.745447 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-d6dd8599-flvk5" podStartSLOduration=1.343112099 podStartE2EDuration="1.745430155s" totalImagesPullingTime="402.318056ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 04:52:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 04:52:13.744281821 +0000 UTC m=+769.086794715" watchObservedRunningTime="2026-07-23 04:52:13.745430155 +0000 UTC m=+769.087943042" Jul 23 04:52:42.770557 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:52:42.770519 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 04:52:52.286979 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:52:52.286895 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 04:53:02.379391 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:53:02.379359 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 04:54:25.131547 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:54:25.131520 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vphbq_b3007292-2534-4dc0-82a3-41d6a7d34c0c/ovn-acl-logging/0.log" Jul 23 04:54:25.132544 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:54:25.132523 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vphbq_b3007292-2534-4dc0-82a3-41d6a7d34c0c/ovn-acl-logging/0.log" Jul 23 04:54:40.770475 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:54:40.770434 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 04:54:52.278305 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:54:52.278270 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 04:55:29.766312 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:55:29.766277 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 04:55:37.477608 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:55:37.477564 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 04:55:46.867724 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:55:46.867640 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 04:55:54.697417 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:55:54.697380 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 04:56:03.076795 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:56:03.076756 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 04:56:14.885408 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:56:14.885368 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 04:56:25.778879 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:56:25.778827 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 04:56:35.594321 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:56:35.594284 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 04:56:57.872102 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:56:57.872056 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 04:57:06.553746 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:57:06.553711 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 04:57:29.965242 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:57:29.965200 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 04:57:41.580991 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:57:41.580959 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 04:57:57.263801 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:57:57.263764 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 04:58:06.582184 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:58:06.582145 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 04:58:17.466256 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:58:17.466218 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 04:58:27.061183 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:58:27.061141 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 04:58:36.363784 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:58:36.363750 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 04:59:25.162023 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:59:25.161995 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vphbq_b3007292-2534-4dc0-82a3-41d6a7d34c0c/ovn-acl-logging/0.log" Jul 23 04:59:25.162561 ip-10-0-128-96 kubenswrapper[2562]: I0723 04:59:25.162241 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vphbq_b3007292-2534-4dc0-82a3-41d6a7d34c0c/ovn-acl-logging/0.log" Jul 23 05:00:23.166334 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:00:23.166295 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 05:00:35.369184 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:00:35.369147 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 05:00:43.971019 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:00:43.970985 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 05:00:56.169571 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:00:56.169534 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 05:01:06.781169 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:01:06.781129 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 05:01:17.888654 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:01:17.888621 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 05:01:26.672975 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:01:26.672939 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 05:01:37.874880 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:01:37.874824 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 05:01:48.766114 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:01:48.766028 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 05:01:57.662491 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:01:57.662452 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 05:02:08.465146 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:02:08.465108 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 05:02:32.864494 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:02:32.864458 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 05:02:39.594504 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:02:39.594470 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw"] Jul 23 05:02:39.597851 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:02:39.597818 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw" Jul 23 05:02:39.603150 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:02:39.603129 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-37d70156-kserve-self-signed-certs\"" Jul 23 05:02:39.624877 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:02:39.624848 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw"] Jul 23 05:02:39.736719 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:02:39.736686 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af0c39f1-cbe9-461a-8bbe-cc8d7df81361-model-cache\") pod \"e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw\" (UID: \"af0c39f1-cbe9-461a-8bbe-cc8d7df81361\") " pod="llm/e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw" Jul 23 05:02:39.736916 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:02:39.736739 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af0c39f1-cbe9-461a-8bbe-cc8d7df81361-tls-certs\") pod \"e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw\" (UID: \"af0c39f1-cbe9-461a-8bbe-cc8d7df81361\") " pod="llm/e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw" Jul 23 05:02:39.736916 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:02:39.736788 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af0c39f1-cbe9-461a-8bbe-cc8d7df81361-dshm\") pod \"e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw\" (UID: \"af0c39f1-cbe9-461a-8bbe-cc8d7df81361\") " pod="llm/e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw" Jul 23 05:02:39.736916 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:02:39.736814 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpr7d\" (UniqueName: \"kubernetes.io/projected/af0c39f1-cbe9-461a-8bbe-cc8d7df81361-kube-api-access-cpr7d\") pod \"e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw\" (UID: \"af0c39f1-cbe9-461a-8bbe-cc8d7df81361\") " pod="llm/e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw" Jul 23 05:02:39.736916 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:02:39.736862 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af0c39f1-cbe9-461a-8bbe-cc8d7df81361-tmp-dir\") pod \"e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw\" (UID: \"af0c39f1-cbe9-461a-8bbe-cc8d7df81361\") " pod="llm/e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw" Jul 23 05:02:39.736916 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:02:39.736889 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af0c39f1-cbe9-461a-8bbe-cc8d7df81361-home\") pod \"e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw\" (UID: \"af0c39f1-cbe9-461a-8bbe-cc8d7df81361\") " pod="llm/e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw" Jul 23 05:02:39.838123 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:02:39.838084 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpr7d\" (UniqueName: \"kubernetes.io/projected/af0c39f1-cbe9-461a-8bbe-cc8d7df81361-kube-api-access-cpr7d\") pod \"e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw\" (UID: \"af0c39f1-cbe9-461a-8bbe-cc8d7df81361\") " pod="llm/e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw" Jul 23 05:02:39.838303 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:02:39.838133 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af0c39f1-cbe9-461a-8bbe-cc8d7df81361-tmp-dir\") pod \"e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw\" (UID: \"af0c39f1-cbe9-461a-8bbe-cc8d7df81361\") " pod="llm/e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw" Jul 23 05:02:39.838303 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:02:39.838169 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af0c39f1-cbe9-461a-8bbe-cc8d7df81361-home\") pod \"e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw\" (UID: \"af0c39f1-cbe9-461a-8bbe-cc8d7df81361\") " pod="llm/e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw" Jul 23 05:02:39.838303 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:02:39.838228 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af0c39f1-cbe9-461a-8bbe-cc8d7df81361-model-cache\") pod \"e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw\" (UID: \"af0c39f1-cbe9-461a-8bbe-cc8d7df81361\") " pod="llm/e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw" Jul 23 05:02:39.838303 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:02:39.838285 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af0c39f1-cbe9-461a-8bbe-cc8d7df81361-tls-certs\") pod \"e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw\" (UID: \"af0c39f1-cbe9-461a-8bbe-cc8d7df81361\") " pod="llm/e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw" Jul 23 05:02:39.838505 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:02:39.838323 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af0c39f1-cbe9-461a-8bbe-cc8d7df81361-dshm\") pod \"e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw\" (UID: \"af0c39f1-cbe9-461a-8bbe-cc8d7df81361\") " pod="llm/e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw" Jul 23 05:02:39.839071 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:02:39.838889 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af0c39f1-cbe9-461a-8bbe-cc8d7df81361-home\") pod \"e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw\" (UID: \"af0c39f1-cbe9-461a-8bbe-cc8d7df81361\") " pod="llm/e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw" Jul 23 05:02:39.839071 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:02:39.838992 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af0c39f1-cbe9-461a-8bbe-cc8d7df81361-tmp-dir\") pod \"e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw\" (UID: \"af0c39f1-cbe9-461a-8bbe-cc8d7df81361\") " pod="llm/e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw" Jul 23 05:02:39.839229 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:02:39.839130 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af0c39f1-cbe9-461a-8bbe-cc8d7df81361-model-cache\") pod \"e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw\" (UID: \"af0c39f1-cbe9-461a-8bbe-cc8d7df81361\") " pod="llm/e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw" Jul 23 05:02:39.844600 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:02:39.841622 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af0c39f1-cbe9-461a-8bbe-cc8d7df81361-dshm\") pod \"e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw\" (UID: \"af0c39f1-cbe9-461a-8bbe-cc8d7df81361\") " pod="llm/e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw" Jul 23 05:02:39.844600 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:02:39.841793 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af0c39f1-cbe9-461a-8bbe-cc8d7df81361-tls-certs\") pod \"e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw\" (UID: \"af0c39f1-cbe9-461a-8bbe-cc8d7df81361\") " pod="llm/e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw" Jul 23 05:02:39.851165 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:02:39.851146 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpr7d\" (UniqueName: \"kubernetes.io/projected/af0c39f1-cbe9-461a-8bbe-cc8d7df81361-kube-api-access-cpr7d\") pod \"e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw\" (UID: \"af0c39f1-cbe9-461a-8bbe-cc8d7df81361\") " pod="llm/e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw" Jul 23 05:02:39.906507 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:02:39.906469 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw" Jul 23 05:02:40.048339 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:02:40.048315 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw"] Jul 23 05:02:40.050495 ip-10-0-128-96 kubenswrapper[2562]: W0723 05:02:40.050467 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaf0c39f1_cbe9_461a_8bbe_cc8d7df81361.slice/crio-a8c828b7ff45d4877820d042021a43bffc98550bff5b12e5bc3401b6d1c9b777 WatchSource:0}: Error finding container a8c828b7ff45d4877820d042021a43bffc98550bff5b12e5bc3401b6d1c9b777: Status 404 returned error can't find the container with id a8c828b7ff45d4877820d042021a43bffc98550bff5b12e5bc3401b6d1c9b777 Jul 23 05:02:40.470114 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:02:40.470020 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 05:02:40.660290 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:02:40.660254 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw" event={"ID":"af0c39f1-cbe9-461a-8bbe-cc8d7df81361","Type":"ContainerStarted","Data":"0e66d175eda5976cb7f23fdc94e48a3826dccad97bc97a3d6bdb388f47eb1ba2"} Jul 23 05:02:40.660290 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:02:40.660294 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw" event={"ID":"af0c39f1-cbe9-461a-8bbe-cc8d7df81361","Type":"ContainerStarted","Data":"a8c828b7ff45d4877820d042021a43bffc98550bff5b12e5bc3401b6d1c9b777"} Jul 23 05:02:40.682390 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:02:40.682340 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw" podStartSLOduration=1.682325009 podStartE2EDuration="1.682325009s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 05:02:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 05:02:40.68052773 +0000 UTC m=+1396.023040620" watchObservedRunningTime="2026-07-23 05:02:40.682325009 +0000 UTC m=+1396.024837951" Jul 23 05:02:49.907057 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:02:49.907020 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw" Jul 23 05:02:49.907516 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:02:49.907072 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw" Jul 23 05:02:49.919692 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:02:49.919667 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw" Jul 23 05:02:50.674988 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:02:50.674947 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 05:02:50.718582 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:02:50.718555 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw" Jul 23 05:03:00.675158 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:03:00.675104 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 05:03:52.868828 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:03:52.868795 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw"] Jul 23 05:03:56.665908 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:03:56.665870 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 05:04:07.888572 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:04:07.888512 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw" podUID="af0c39f1-cbe9-461a-8bbe-cc8d7df81361" containerName="main" containerID="cri-o://0e66d175eda5976cb7f23fdc94e48a3826dccad97bc97a3d6bdb388f47eb1ba2" gracePeriod=45 Jul 23 05:04:08.076008 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:04:08.075971 2562 generic.go:356] "Generic (PLEG): container finished" podID="af0c39f1-cbe9-461a-8bbe-cc8d7df81361" containerID="0e66d175eda5976cb7f23fdc94e48a3826dccad97bc97a3d6bdb388f47eb1ba2" exitCode=0 Jul 23 05:04:08.076134 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:04:08.076022 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw" event={"ID":"af0c39f1-cbe9-461a-8bbe-cc8d7df81361","Type":"ContainerDied","Data":"0e66d175eda5976cb7f23fdc94e48a3826dccad97bc97a3d6bdb388f47eb1ba2"} Jul 23 05:04:08.133133 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:04:08.133111 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw" Jul 23 05:04:08.281608 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:04:08.281516 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af0c39f1-cbe9-461a-8bbe-cc8d7df81361-home\") pod \"af0c39f1-cbe9-461a-8bbe-cc8d7df81361\" (UID: \"af0c39f1-cbe9-461a-8bbe-cc8d7df81361\") " Jul 23 05:04:08.281608 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:04:08.281601 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpr7d\" (UniqueName: \"kubernetes.io/projected/af0c39f1-cbe9-461a-8bbe-cc8d7df81361-kube-api-access-cpr7d\") pod \"af0c39f1-cbe9-461a-8bbe-cc8d7df81361\" (UID: \"af0c39f1-cbe9-461a-8bbe-cc8d7df81361\") " Jul 23 05:04:08.281825 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:04:08.281650 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af0c39f1-cbe9-461a-8bbe-cc8d7df81361-tls-certs\") pod \"af0c39f1-cbe9-461a-8bbe-cc8d7df81361\" (UID: \"af0c39f1-cbe9-461a-8bbe-cc8d7df81361\") " Jul 23 05:04:08.281825 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:04:08.281681 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af0c39f1-cbe9-461a-8bbe-cc8d7df81361-model-cache\") pod \"af0c39f1-cbe9-461a-8bbe-cc8d7df81361\" (UID: \"af0c39f1-cbe9-461a-8bbe-cc8d7df81361\") " Jul 23 05:04:08.281825 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:04:08.281708 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af0c39f1-cbe9-461a-8bbe-cc8d7df81361-dshm\") pod \"af0c39f1-cbe9-461a-8bbe-cc8d7df81361\" (UID: \"af0c39f1-cbe9-461a-8bbe-cc8d7df81361\") " Jul 23 05:04:08.281825 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:04:08.281725 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af0c39f1-cbe9-461a-8bbe-cc8d7df81361-tmp-dir\") pod \"af0c39f1-cbe9-461a-8bbe-cc8d7df81361\" (UID: \"af0c39f1-cbe9-461a-8bbe-cc8d7df81361\") " Jul 23 05:04:08.282068 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:04:08.281822 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af0c39f1-cbe9-461a-8bbe-cc8d7df81361-home" (OuterVolumeSpecName: "home") pod "af0c39f1-cbe9-461a-8bbe-cc8d7df81361" (UID: "af0c39f1-cbe9-461a-8bbe-cc8d7df81361"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 05:04:08.282068 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:04:08.282023 2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af0c39f1-cbe9-461a-8bbe-cc8d7df81361-home\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 05:04:08.282068 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:04:08.282060 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af0c39f1-cbe9-461a-8bbe-cc8d7df81361-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "af0c39f1-cbe9-461a-8bbe-cc8d7df81361" (UID: "af0c39f1-cbe9-461a-8bbe-cc8d7df81361"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 05:04:08.282210 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:04:08.282052 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af0c39f1-cbe9-461a-8bbe-cc8d7df81361-model-cache" (OuterVolumeSpecName: "model-cache") pod "af0c39f1-cbe9-461a-8bbe-cc8d7df81361" (UID: "af0c39f1-cbe9-461a-8bbe-cc8d7df81361"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 05:04:08.283946 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:04:08.283919 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af0c39f1-cbe9-461a-8bbe-cc8d7df81361-dshm" (OuterVolumeSpecName: "dshm") pod "af0c39f1-cbe9-461a-8bbe-cc8d7df81361" (UID: "af0c39f1-cbe9-461a-8bbe-cc8d7df81361"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 05:04:08.284055 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:04:08.283976 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af0c39f1-cbe9-461a-8bbe-cc8d7df81361-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "af0c39f1-cbe9-461a-8bbe-cc8d7df81361" (UID: "af0c39f1-cbe9-461a-8bbe-cc8d7df81361"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 05:04:08.284055 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:04:08.283988 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af0c39f1-cbe9-461a-8bbe-cc8d7df81361-kube-api-access-cpr7d" (OuterVolumeSpecName: "kube-api-access-cpr7d") pod "af0c39f1-cbe9-461a-8bbe-cc8d7df81361" (UID: "af0c39f1-cbe9-461a-8bbe-cc8d7df81361"). InnerVolumeSpecName "kube-api-access-cpr7d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 05:04:08.382733 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:04:08.382694 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpr7d\" (UniqueName: \"kubernetes.io/projected/af0c39f1-cbe9-461a-8bbe-cc8d7df81361-kube-api-access-cpr7d\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 05:04:08.382910 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:04:08.382737 2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af0c39f1-cbe9-461a-8bbe-cc8d7df81361-tls-certs\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 05:04:08.382910 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:04:08.382759 2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af0c39f1-cbe9-461a-8bbe-cc8d7df81361-model-cache\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 05:04:08.382910 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:04:08.382770 2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af0c39f1-cbe9-461a-8bbe-cc8d7df81361-dshm\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 05:04:08.382910 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:04:08.382781 2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af0c39f1-cbe9-461a-8bbe-cc8d7df81361-tmp-dir\") on node \"ip-10-0-128-96.ec2.internal\" DevicePath \"\"" Jul 23 05:04:09.082518 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:04:09.082493 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw" Jul 23 05:04:09.082518 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:04:09.082511 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw" event={"ID":"af0c39f1-cbe9-461a-8bbe-cc8d7df81361","Type":"ContainerDied","Data":"a8c828b7ff45d4877820d042021a43bffc98550bff5b12e5bc3401b6d1c9b777"} Jul 23 05:04:09.083120 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:04:09.082557 2562 scope.go:117] "RemoveContainer" containerID="0e66d175eda5976cb7f23fdc94e48a3826dccad97bc97a3d6bdb388f47eb1ba2" Jul 23 05:04:09.104868 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:04:09.104827 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw"] Jul 23 05:04:09.107130 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:04:09.107110 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-37d70156-kserve-67b88c5999-4zgmw"] Jul 23 05:04:09.207807 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:04:09.207783 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af0c39f1-cbe9-461a-8bbe-cc8d7df81361" path="/var/lib/kubelet/pods/af0c39f1-cbe9-461a-8bbe-cc8d7df81361/volumes" Jul 23 05:04:10.963246 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:04:10.963205 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 05:04:17.971750 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:04:17.971713 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 05:04:25.190167 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:04:25.190140 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vphbq_b3007292-2534-4dc0-82a3-41d6a7d34c0c/ovn-acl-logging/0.log" Jul 23 05:04:25.190632 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:04:25.190150 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vphbq_b3007292-2534-4dc0-82a3-41d6a7d34c0c/ovn-acl-logging/0.log" Jul 23 05:04:34.471315 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:04:34.471284 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 05:04:43.869202 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:04:43.869163 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 05:04:53.266716 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:04:53.266677 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 05:05:03.260207 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:05:03.260170 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 05:05:13.967942 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:05:13.967885 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 05:05:23.774603 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:05:23.774566 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 05:05:36.862864 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:05:36.862814 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 05:05:45.569791 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:05:45.569756 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 05:05:54.764376 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:05:54.764331 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 05:06:05.658523 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:06:05.658487 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 05:06:16.658467 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:06:16.658390 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 05:06:22.259894 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:06:22.259858 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 05:06:39.253202 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:06:39.253160 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 05:06:50.069628 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:06:50.069591 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 05:06:58.856594 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:06:58.856549 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 05:07:07.356294 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:07.356246 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nvp4j"] Jul 23 05:07:11.130566 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:11.130532 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-d6dd8599-flvk5_97b0d574-ea8d-4bc0-a1f8-16cf95c45572/authorino/0.log" Jul 23 05:07:17.274286 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:17.274256 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6576c599b-jw2bk_4c1ec1df-d68f-4e79-9463-c4a6f58629ec/copy-manifests/0.log" Jul 23 05:07:17.282555 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:17.282525 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6576c599b-jw2bk_4c1ec1df-d68f-4e79-9463-c4a6f58629ec/manager/0.log" Jul 23 05:07:19.291978 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:19.291936 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-d6dd8599-flvk5_97b0d574-ea8d-4bc0-a1f8-16cf95c45572/authorino/0.log" Jul 23 05:07:19.629482 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:19.629444 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-8srbc_def40d9d-1587-4886-b14e-7c114f255e6e/kuadrant-console-plugin/0.log" Jul 23 05:07:20.000305 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:20.000215 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-nvp4j_4c4a8f86-e655-46fe-9371-f9a6f10501dc/limitador/0.log" Jul 23 05:07:20.816556 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:20.816530 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7f5c59f4bf-xkxqh_28e63d2c-7c59-4d3b-bce8-0b2ad7aeaf09/kube-auth-proxy/0.log" Jul 23 05:07:21.294627 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:21.294590 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-657f4d9d48-7l4rd_244b6204-6371-4924-815e-fd84477ce28d/router/0.log" Jul 23 05:07:21.972420 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:21.972386 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmb7xg_4b48eb5e-522f-4e02-9bb0-902f172b3a06/main/0.log" Jul 23 05:07:22.091232 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:22.091192 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-8vgln_fe4557e0-dad6-477e-b318-81579698c848/main/0.log" Jul 23 05:07:22.206145 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:22.206111 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-thp2s_d5777220-879c-4109-af66-cb99d923565e/main/0.log" Jul 23 05:07:30.285727 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:30.285696 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-cbzpv_555665cd-6364-4d9f-8326-4973b140cc64/global-pull-secret-syncer/0.log" Jul 23 05:07:30.502825 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:30.502788 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-zhq6d_ba17063f-a4cc-4378-afe1-6b4aad36adef/konnectivity-agent/0.log" Jul 23 05:07:30.525775 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:30.525742 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-96.ec2.internal_5666b1587a613009c403213e61aac89c/haproxy/0.log" Jul 23 05:07:35.678784 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:35.678687 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-d6dd8599-flvk5_97b0d574-ea8d-4bc0-a1f8-16cf95c45572/authorino/0.log" Jul 23 05:07:35.778327 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:35.778287 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-8srbc_def40d9d-1587-4886-b14e-7c114f255e6e/kuadrant-console-plugin/0.log" Jul 23 05:07:35.919022 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:35.918989 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-nvp4j_4c4a8f86-e655-46fe-9371-f9a6f10501dc/limitador/0.log" Jul 23 05:07:37.778040 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:37.778008 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-m5fv5_3809e4cf-5052-4402-ada4-7668bdf98a74/kube-state-metrics/0.log" Jul 23 05:07:37.795947 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:37.795917 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-m5fv5_3809e4cf-5052-4402-ada4-7668bdf98a74/kube-rbac-proxy-main/0.log" Jul 23 05:07:37.824438 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:37.824409 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-m5fv5_3809e4cf-5052-4402-ada4-7668bdf98a74/kube-rbac-proxy-self/0.log" Jul 23 05:07:37.854891 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:37.854856 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6d98b9888d-kvnzs_330ee043-0fca-4da8-b619-3853fcfeb19b/metrics-server/0.log" Jul 23 05:07:37.927536 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:37.927507 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6pnvh_a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc/node-exporter/0.log" Jul 23 05:07:37.947496 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:37.947470 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6pnvh_a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc/kube-rbac-proxy/0.log" Jul 23 05:07:37.968395 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:37.968371 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6pnvh_a3dd09ae-2af3-4c1d-985f-f588c6ab7fbc/init-textfile/0.log" Jul 23 05:07:38.178658 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:38.178626 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-tk424_49aa1a49-bb25-47e7-9f6c-686e559571f7/kube-rbac-proxy-main/0.log" Jul 23 05:07:38.202470 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:38.202445 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-tk424_49aa1a49-bb25-47e7-9f6c-686e559571f7/kube-rbac-proxy-self/0.log" Jul 23 05:07:38.225424 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:38.225395 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-tk424_49aa1a49-bb25-47e7-9f6c-686e559571f7/openshift-state-metrics/0.log" Jul 23 05:07:38.272675 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:38.272623 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7ace0723-60be-4fd5-8251-2706d257e9f9/prometheus/0.log" Jul 23 05:07:38.293501 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:38.293473 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7ace0723-60be-4fd5-8251-2706d257e9f9/config-reloader/0.log" Jul 23 05:07:38.315668 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:38.315632 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7ace0723-60be-4fd5-8251-2706d257e9f9/thanos-sidecar/0.log" Jul 23 05:07:38.337580 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:38.337550 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7ace0723-60be-4fd5-8251-2706d257e9f9/kube-rbac-proxy-web/0.log" Jul 23 05:07:38.359563 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:38.359534 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7ace0723-60be-4fd5-8251-2706d257e9f9/kube-rbac-proxy/0.log" Jul 23 05:07:38.381990 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:38.381962 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7ace0723-60be-4fd5-8251-2706d257e9f9/kube-rbac-proxy-thanos/0.log" Jul 23 05:07:38.411696 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:38.411666 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7ace0723-60be-4fd5-8251-2706d257e9f9/init-config-reloader/0.log" Jul 23 05:07:38.442508 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:38.442375 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-jpzmn_3d7f6ab8-9a28-4631-a93c-726bf5ed7e18/prometheus-operator/0.log" Jul 23 05:07:38.459934 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:38.459910 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-jpzmn_3d7f6ab8-9a28-4631-a93c-726bf5ed7e18/kube-rbac-proxy/0.log" Jul 23 05:07:38.604820 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:38.604782 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-765ff6dd64-cql22_aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e/thanos-query/0.log" Jul 23 05:07:38.628102 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:38.628066 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-765ff6dd64-cql22_aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e/kube-rbac-proxy-web/0.log" Jul 23 05:07:38.649167 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:38.649137 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-765ff6dd64-cql22_aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e/kube-rbac-proxy/0.log" Jul 23 05:07:38.671306 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:38.671277 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-765ff6dd64-cql22_aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e/prom-label-proxy/0.log" Jul 23 05:07:38.692249 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:38.692226 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-765ff6dd64-cql22_aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e/kube-rbac-proxy-rules/0.log" Jul 23 05:07:38.714089 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:38.714009 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-765ff6dd64-cql22_aa2ca5d6-9878-4b4f-9e8d-c0288f40d18e/kube-rbac-proxy-metrics/0.log" Jul 23 05:07:40.568000 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:40.567969 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gks9x/perf-node-gather-daemonset-dkp48"] Jul 23 05:07:40.568384 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:40.568345 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af0c39f1-cbe9-461a-8bbe-cc8d7df81361" containerName="main" Jul 23 05:07:40.568384 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:40.568356 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="af0c39f1-cbe9-461a-8bbe-cc8d7df81361" containerName="main" Jul 23 05:07:40.568456 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:40.568413 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="af0c39f1-cbe9-461a-8bbe-cc8d7df81361" containerName="main" Jul 23 05:07:40.571669 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:40.571648 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gks9x/perf-node-gather-daemonset-dkp48" Jul 23 05:07:40.573946 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:40.573927 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gks9x\"/\"openshift-service-ca.crt\"" Jul 23 05:07:40.574066 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:40.573928 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gks9x\"/\"default-dockercfg-2lcg7\"" Jul 23 05:07:40.574818 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:40.574798 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gks9x\"/\"kube-root-ca.crt\"" Jul 23 05:07:40.585577 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:40.585548 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gks9x/perf-node-gather-daemonset-dkp48"] Jul 23 05:07:40.661747 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:40.661711 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9ccfa75c-7059-486a-abe6-b0f51bb66c98-podres\") pod \"perf-node-gather-daemonset-dkp48\" (UID: \"9ccfa75c-7059-486a-abe6-b0f51bb66c98\") " pod="openshift-must-gather-gks9x/perf-node-gather-daemonset-dkp48" Jul 23 05:07:40.661978 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:40.661770 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9ccfa75c-7059-486a-abe6-b0f51bb66c98-sys\") pod \"perf-node-gather-daemonset-dkp48\" (UID: \"9ccfa75c-7059-486a-abe6-b0f51bb66c98\") " pod="openshift-must-gather-gks9x/perf-node-gather-daemonset-dkp48" Jul 23 05:07:40.661978 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:40.661884 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9ccfa75c-7059-486a-abe6-b0f51bb66c98-proc\") pod \"perf-node-gather-daemonset-dkp48\" (UID: \"9ccfa75c-7059-486a-abe6-b0f51bb66c98\") " pod="openshift-must-gather-gks9x/perf-node-gather-daemonset-dkp48" Jul 23 05:07:40.661978 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:40.661933 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9ccfa75c-7059-486a-abe6-b0f51bb66c98-lib-modules\") pod \"perf-node-gather-daemonset-dkp48\" (UID: \"9ccfa75c-7059-486a-abe6-b0f51bb66c98\") " pod="openshift-must-gather-gks9x/perf-node-gather-daemonset-dkp48" Jul 23 05:07:40.662085 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:40.661985 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdsnm\" (UniqueName: \"kubernetes.io/projected/9ccfa75c-7059-486a-abe6-b0f51bb66c98-kube-api-access-kdsnm\") pod \"perf-node-gather-daemonset-dkp48\" (UID: \"9ccfa75c-7059-486a-abe6-b0f51bb66c98\") " pod="openshift-must-gather-gks9x/perf-node-gather-daemonset-dkp48" Jul 23 05:07:40.763158 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:40.763107 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9ccfa75c-7059-486a-abe6-b0f51bb66c98-sys\") pod \"perf-node-gather-daemonset-dkp48\" (UID: \"9ccfa75c-7059-486a-abe6-b0f51bb66c98\") " pod="openshift-must-gather-gks9x/perf-node-gather-daemonset-dkp48" Jul 23 05:07:40.763358 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:40.763181 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9ccfa75c-7059-486a-abe6-b0f51bb66c98-proc\") pod \"perf-node-gather-daemonset-dkp48\" (UID: \"9ccfa75c-7059-486a-abe6-b0f51bb66c98\") " pod="openshift-must-gather-gks9x/perf-node-gather-daemonset-dkp48" Jul 23 05:07:40.763358 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:40.763218 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9ccfa75c-7059-486a-abe6-b0f51bb66c98-lib-modules\") pod \"perf-node-gather-daemonset-dkp48\" (UID: \"9ccfa75c-7059-486a-abe6-b0f51bb66c98\") " pod="openshift-must-gather-gks9x/perf-node-gather-daemonset-dkp48" Jul 23 05:07:40.763358 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:40.763233 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9ccfa75c-7059-486a-abe6-b0f51bb66c98-sys\") pod \"perf-node-gather-daemonset-dkp48\" (UID: \"9ccfa75c-7059-486a-abe6-b0f51bb66c98\") " pod="openshift-must-gather-gks9x/perf-node-gather-daemonset-dkp48" Jul 23 05:07:40.763358 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:40.763272 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdsnm\" (UniqueName: \"kubernetes.io/projected/9ccfa75c-7059-486a-abe6-b0f51bb66c98-kube-api-access-kdsnm\") pod \"perf-node-gather-daemonset-dkp48\" (UID: \"9ccfa75c-7059-486a-abe6-b0f51bb66c98\") " pod="openshift-must-gather-gks9x/perf-node-gather-daemonset-dkp48" Jul 23 05:07:40.763358 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:40.763304 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9ccfa75c-7059-486a-abe6-b0f51bb66c98-proc\") pod \"perf-node-gather-daemonset-dkp48\" (UID: \"9ccfa75c-7059-486a-abe6-b0f51bb66c98\") " pod="openshift-must-gather-gks9x/perf-node-gather-daemonset-dkp48" Jul 23 05:07:40.763358 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:40.763342 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9ccfa75c-7059-486a-abe6-b0f51bb66c98-podres\") pod \"perf-node-gather-daemonset-dkp48\" (UID: \"9ccfa75c-7059-486a-abe6-b0f51bb66c98\") " pod="openshift-must-gather-gks9x/perf-node-gather-daemonset-dkp48" Jul 23 05:07:40.763358 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:40.763354 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9ccfa75c-7059-486a-abe6-b0f51bb66c98-lib-modules\") pod \"perf-node-gather-daemonset-dkp48\" (UID: \"9ccfa75c-7059-486a-abe6-b0f51bb66c98\") " pod="openshift-must-gather-gks9x/perf-node-gather-daemonset-dkp48" Jul 23 05:07:40.763584 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:40.763438 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9ccfa75c-7059-486a-abe6-b0f51bb66c98-podres\") pod \"perf-node-gather-daemonset-dkp48\" (UID: \"9ccfa75c-7059-486a-abe6-b0f51bb66c98\") " pod="openshift-must-gather-gks9x/perf-node-gather-daemonset-dkp48" Jul 23 05:07:40.775007 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:40.774972 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdsnm\" (UniqueName: \"kubernetes.io/projected/9ccfa75c-7059-486a-abe6-b0f51bb66c98-kube-api-access-kdsnm\") pod \"perf-node-gather-daemonset-dkp48\" (UID: \"9ccfa75c-7059-486a-abe6-b0f51bb66c98\") " pod="openshift-must-gather-gks9x/perf-node-gather-daemonset-dkp48" Jul 23 05:07:40.881600 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:40.881548 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gks9x/perf-node-gather-daemonset-dkp48" Jul 23 05:07:41.017753 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:41.017687 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gks9x/perf-node-gather-daemonset-dkp48"] Jul 23 05:07:41.020503 ip-10-0-128-96 kubenswrapper[2562]: W0723 05:07:41.020478 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9ccfa75c_7059_486a_abe6_b0f51bb66c98.slice/crio-26f45e39f355cdb9a4063230135e1eedd13aba3223cb075684e5e57f0997f656 WatchSource:0}: Error finding container 26f45e39f355cdb9a4063230135e1eedd13aba3223cb075684e5e57f0997f656: Status 404 returned error can't find the container with id 26f45e39f355cdb9a4063230135e1eedd13aba3223cb075684e5e57f0997f656 Jul 23 05:07:41.085376 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:41.085342 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gks9x/perf-node-gather-daemonset-dkp48" event={"ID":"9ccfa75c-7059-486a-abe6-b0f51bb66c98","Type":"ContainerStarted","Data":"26f45e39f355cdb9a4063230135e1eedd13aba3223cb075684e5e57f0997f656"} Jul 23 05:07:41.232412 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:41.232383 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-pzvsl_1a3d4f1d-65d6-4abd-a649-6353217d4c77/download-server/0.log" Jul 23 05:07:41.812694 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:41.812632 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-4p4x8_d8777622-488c-4307-a8fc-5d4af6137595/volume-data-source-validator/0.log" Jul 23 05:07:42.097001 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:42.096961 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gks9x/perf-node-gather-daemonset-dkp48" event={"ID":"9ccfa75c-7059-486a-abe6-b0f51bb66c98","Type":"ContainerStarted","Data":"a864f517a4eca395e140f51ece735ce7ac69402df6418aff294a112164fe5db0"} Jul 23 05:07:42.097183 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:42.097071 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gks9x/perf-node-gather-daemonset-dkp48" Jul 23 05:07:42.114873 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:42.114803 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gks9x/perf-node-gather-daemonset-dkp48" podStartSLOduration=2.114783569 podStartE2EDuration="2.114783569s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 05:07:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 05:07:42.112779892 +0000 UTC m=+1697.455292780" watchObservedRunningTime="2026-07-23 05:07:42.114783569 +0000 UTC m=+1697.457296458" Jul 23 05:07:42.854274 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:42.854247 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-45c4t_10491e7a-c5ab-4d40-a5ff-30d5714ffdf9/dns/0.log" Jul 23 05:07:42.876851 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:42.876809 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-45c4t_10491e7a-c5ab-4d40-a5ff-30d5714ffdf9/kube-rbac-proxy/0.log" Jul 23 05:07:43.026189 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:43.026161 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-57bmk_b9a49f0e-338a-47ac-90f7-281dae5538e4/dns-node-resolver/0.log" Jul 23 05:07:43.593907 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:43.593871 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5f446fbc9d-hzvct_d21b024b-02b0-4c6a-8d5c-36a20694fea7/registry/0.log" Jul 23 05:07:43.616647 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:43.616617 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-grlbc_d69de93b-d93f-4f10-b618-0cdcb545197a/node-ca/0.log" Jul 23 05:07:44.719417 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:44.719382 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7f5c59f4bf-xkxqh_28e63d2c-7c59-4d3b-bce8-0b2ad7aeaf09/kube-auth-proxy/0.log" Jul 23 05:07:44.862482 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:44.862447 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-657f4d9d48-7l4rd_244b6204-6371-4924-815e-fd84477ce28d/router/0.log" Jul 23 05:07:45.529010 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:45.528926 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qj5zw_8fb208f1-6626-4be0-9bfa-7cb8766d58ec/serve-healthcheck-canary/0.log" Jul 23 05:07:46.013215 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:46.013177 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-8v8xp_aa0558ac-8329-4a03-8a88-76a2a42d49a1/insights-operator/1.log" Jul 23 05:07:46.013714 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:46.013331 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-8v8xp_aa0558ac-8329-4a03-8a88-76a2a42d49a1/insights-operator/0.log" Jul 23 05:07:46.036752 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:46.036724 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6k5dg_b2a0526d-c1cd-47da-95e8-aaee5f6f094d/kube-rbac-proxy/0.log" Jul 23 05:07:46.059129 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:46.059105 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6k5dg_b2a0526d-c1cd-47da-95e8-aaee5f6f094d/exporter/0.log" Jul 23 05:07:46.082570 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:46.082542 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6k5dg_b2a0526d-c1cd-47da-95e8-aaee5f6f094d/extractor/0.log" Jul 23 05:07:48.112728 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:48.112692 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gks9x/perf-node-gather-daemonset-dkp48" Jul 23 05:07:48.673671 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:48.673639 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6576c599b-jw2bk_4c1ec1df-d68f-4e79-9463-c4a6f58629ec/manager/0.log" Jul 23 05:07:48.697543 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:48.697513 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6576c599b-jw2bk_4c1ec1df-d68f-4e79-9463-c4a6f58629ec/copy-manifests/0.log" Jul 23 05:07:50.219792 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:50.219762 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-56ccd69865-qjbwv_18f43204-e1ae-4559-b725-54053a68b3e5/manager/0.log" Jul 23 05:07:51.464753 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:51.464715 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7f5c59f4bf-xkxqh_28e63d2c-7c59-4d3b-bce8-0b2ad7aeaf09/kube-auth-proxy/0.log" Jul 23 05:07:51.573876 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:51.573822 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-657f4d9d48-7l4rd_244b6204-6371-4924-815e-fd84477ce28d/router/0.log" Jul 23 05:07:52.810320 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:52.810225 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7f5c59f4bf-xkxqh_28e63d2c-7c59-4d3b-bce8-0b2ad7aeaf09/kube-auth-proxy/0.log" Jul 23 05:07:52.942918 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:52.942891 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-657f4d9d48-7l4rd_244b6204-6371-4924-815e-fd84477ce28d/router/0.log" Jul 23 05:07:57.279479 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:57.279447 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zzgwf_c49b7ef0-9710-41cd-9d89-145654e92fc6/kube-multus-additional-cni-plugins/0.log" Jul 23 05:07:57.299790 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:57.299753 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zzgwf_c49b7ef0-9710-41cd-9d89-145654e92fc6/egress-router-binary-copy/0.log" Jul 23 05:07:57.320738 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:57.320713 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zzgwf_c49b7ef0-9710-41cd-9d89-145654e92fc6/cni-plugins/0.log" Jul 23 05:07:57.341199 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:57.341168 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zzgwf_c49b7ef0-9710-41cd-9d89-145654e92fc6/bond-cni-plugin/0.log" Jul 23 05:07:57.362083 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:57.362055 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zzgwf_c49b7ef0-9710-41cd-9d89-145654e92fc6/routeoverride-cni/0.log" Jul 23 05:07:57.384574 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:57.384549 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zzgwf_c49b7ef0-9710-41cd-9d89-145654e92fc6/whereabouts-cni-bincopy/0.log" Jul 23 05:07:57.405895 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:57.405865 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zzgwf_c49b7ef0-9710-41cd-9d89-145654e92fc6/whereabouts-cni/0.log" Jul 23 05:07:57.439926 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:57.439897 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-w7d9f_0bf5e605-388c-4f50-8716-c350190d11d9/kube-multus/0.log" Jul 23 05:07:57.503261 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:57.503232 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mpnqv_dab60874-8b3b-411d-b58e-4ec857bc9629/network-metrics-daemon/0.log" Jul 23 05:07:57.521117 ip-10-0-128-96 kubenswrapper[2562]: I0723 05:07:57.521089 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mpnqv_dab60874-8b3b-411d-b58e-4ec857bc9629/kube-rbac-proxy/0.log"