Jul 08 22:12:14.432958 ip-10-0-133-162 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 08 22:12:14.432972 ip-10-0-133-162 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 08 22:12:14.432999 ip-10-0-133-162 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 08 22:12:14.433360 ip-10-0-133-162 systemd[1]: Failed to start Kubernetes Kubelet. Jul 08 22:12:24.504856 ip-10-0-133-162 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 08 22:12:24.504872 ip-10-0-133-162 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 2c8d7ea4e2384396ab79f75880cdab9d -- Jul 08 22:14:26.905224 ip-10-0-133-162 systemd[1]: Starting Kubernetes Kubelet... Jul 08 22:14:29.027137 ip-10-0-133-162 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 22:14:29.027137 ip-10-0-133-162 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 08 22:14:29.027137 ip-10-0-133-162 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 22:14:29.027137 ip-10-0-133-162 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 08 22:14:29.027137 ip-10-0-133-162 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 22:14:29.030628 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.030488 2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 08 22:14:29.036560 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036533 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 22:14:29.036560 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036555 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 22:14:29.036560 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036560 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 22:14:29.036560 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036565 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 22:14:29.036560 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036578 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 22:14:29.036871 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036583 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 22:14:29.036871 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036587 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 08 22:14:29.036871 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036591 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 22:14:29.036871 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036595 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 22:14:29.036871 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036599 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 22:14:29.036871 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036604 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 22:14:29.036871 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036607 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 22:14:29.036871 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036611 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 22:14:29.036871 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036616 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 22:14:29.036871 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036620 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 22:14:29.036871 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036626 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 22:14:29.036871 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036630 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 22:14:29.036871 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036634 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 22:14:29.036871 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036638 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 22:14:29.036871 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036641 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 22:14:29.036871 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036646 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 22:14:29.036871 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036650 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 22:14:29.036871 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036653 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 22:14:29.036871 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036657 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 22:14:29.036871 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036661 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 22:14:29.037658 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036665 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 22:14:29.037658 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036669 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 22:14:29.037658 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036673 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 22:14:29.037658 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036677 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 22:14:29.037658 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036681 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 22:14:29.037658 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036685 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 22:14:29.037658 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036688 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 22:14:29.037658 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036692 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 22:14:29.037658 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036696 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 22:14:29.037658 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036700 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 22:14:29.037658 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036704 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 22:14:29.037658 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036707 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 22:14:29.037658 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036714 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 22:14:29.037658 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036719 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 22:14:29.037658 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036723 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 22:14:29.037658 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036727 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 22:14:29.037658 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036732 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 22:14:29.037658 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036736 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 22:14:29.037658 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036740 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 22:14:29.037658 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036745 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 22:14:29.038479 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036749 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 22:14:29.038479 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036756 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 22:14:29.038479 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036760 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 22:14:29.038479 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036764 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 22:14:29.038479 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036768 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 22:14:29.038479 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036772 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 22:14:29.038479 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036776 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 22:14:29.038479 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036780 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 22:14:29.038479 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036784 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 22:14:29.038479 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036789 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 22:14:29.038479 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036792 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 22:14:29.038479 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036797 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 22:14:29.038479 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036800 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 22:14:29.038479 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036807 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 22:14:29.038479 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036813 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 22:14:29.038479 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036819 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 22:14:29.038479 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036823 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 22:14:29.038479 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036827 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 22:14:29.038479 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036830 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 22:14:29.038479 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036835 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 22:14:29.039074 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036839 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 22:14:29.039074 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036843 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 22:14:29.039074 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036848 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 22:14:29.039074 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036852 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 22:14:29.039074 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036857 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 22:14:29.039074 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036863 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 22:14:29.039074 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036871 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 22:14:29.039074 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036893 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 22:14:29.039074 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036900 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 22:14:29.039074 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036905 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 22:14:29.039074 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036909 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 22:14:29.039074 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036913 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 22:14:29.039074 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036917 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 22:14:29.039074 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036921 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 22:14:29.039074 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036926 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 22:14:29.039074 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036930 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 22:14:29.039074 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036934 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 22:14:29.039074 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036939 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 22:14:29.039074 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036943 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 22:14:29.039521 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036947 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 22:14:29.039521 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036952 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 22:14:29.039521 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036956 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 22:14:29.039521 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036960 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 22:14:29.039521 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036964 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 22:14:29.039521 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036968 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 22:14:29.039521 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036972 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 22:14:29.039521 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036978 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 22:14:29.039521 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036982 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 22:14:29.039521 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036988 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 22:14:29.039521 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036992 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 22:14:29.039521 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.036997 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 22:14:29.039521 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.037002 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 22:14:29.039521 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.037007 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 22:14:29.039521 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.037012 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 22:14:29.039521 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039475 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 22:14:29.039521 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039489 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 22:14:29.039521 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039494 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 22:14:29.039521 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039497 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 22:14:29.039986 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039501 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 22:14:29.039986 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039505 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 22:14:29.039986 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039509 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 22:14:29.039986 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039511 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 22:14:29.039986 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039514 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 22:14:29.039986 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039517 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 22:14:29.039986 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039520 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 22:14:29.039986 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039523 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 22:14:29.039986 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039526 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 22:14:29.039986 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039529 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 22:14:29.039986 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039531 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 22:14:29.039986 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039534 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 22:14:29.039986 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039537 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 22:14:29.039986 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039539 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 22:14:29.039986 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039543 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 22:14:29.039986 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039546 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 22:14:29.039986 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039549 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 22:14:29.039986 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039551 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 22:14:29.039986 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039554 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 22:14:29.039986 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039556 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 22:14:29.040477 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039559 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 22:14:29.040477 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039562 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 22:14:29.040477 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039564 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 22:14:29.040477 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039567 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 22:14:29.040477 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039569 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 22:14:29.040477 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039572 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 22:14:29.040477 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039574 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 22:14:29.040477 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039577 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 22:14:29.040477 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039580 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 22:14:29.040477 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039582 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 22:14:29.040477 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039585 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 22:14:29.040477 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039587 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 22:14:29.040477 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039590 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 22:14:29.040477 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039593 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 22:14:29.040477 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039596 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 22:14:29.040477 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039598 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 22:14:29.040477 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039601 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 22:14:29.040477 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039604 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 22:14:29.040477 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039606 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 22:14:29.040477 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039609 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 22:14:29.041110 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039612 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 22:14:29.041110 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039614 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 22:14:29.041110 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039617 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 22:14:29.041110 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039619 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 22:14:29.041110 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039622 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 22:14:29.041110 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039625 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 22:14:29.041110 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039628 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 22:14:29.041110 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039632 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 22:14:29.041110 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039635 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 22:14:29.041110 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039638 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 22:14:29.041110 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039641 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 22:14:29.041110 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039643 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 22:14:29.041110 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039646 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 08 22:14:29.041110 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039648 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 22:14:29.041110 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039650 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 22:14:29.041110 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039653 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 22:14:29.041110 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039655 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 22:14:29.041110 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039658 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 22:14:29.041110 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039660 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 22:14:29.041600 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039663 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 22:14:29.041600 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039665 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 22:14:29.041600 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039668 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 22:14:29.041600 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039670 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 22:14:29.041600 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039673 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 22:14:29.041600 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039676 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 22:14:29.041600 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039680 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 22:14:29.041600 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039683 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 22:14:29.041600 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039686 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 22:14:29.041600 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039689 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 22:14:29.041600 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039691 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 22:14:29.041600 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039694 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 22:14:29.041600 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039696 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 22:14:29.041600 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039699 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 22:14:29.041600 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039702 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 22:14:29.041600 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039704 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 22:14:29.041600 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039707 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 22:14:29.041600 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039709 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 22:14:29.041600 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039712 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 22:14:29.042068 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039715 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 22:14:29.042068 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039718 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 22:14:29.042068 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039721 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 22:14:29.042068 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039723 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 22:14:29.042068 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039726 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 22:14:29.042068 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039729 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 22:14:29.042068 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039731 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 22:14:29.042068 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039734 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 22:14:29.042068 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039736 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 22:14:29.042068 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039739 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 22:14:29.042068 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039741 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 22:14:29.042068 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039744 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 22:14:29.042068 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039746 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 22:14:29.042068 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039748 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 22:14:29.042068 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039751 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 22:14:29.042068 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039753 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 22:14:29.042068 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.039756 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 22:14:29.042068 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040185 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 22:14:29.042068 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040190 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 22:14:29.042068 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040193 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 22:14:29.042559 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040196 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 22:14:29.042559 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040199 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 22:14:29.042559 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040202 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 22:14:29.042559 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040204 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 22:14:29.042559 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040207 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 22:14:29.042559 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040210 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 22:14:29.042559 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040212 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 22:14:29.042559 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040215 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 22:14:29.042559 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040217 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 22:14:29.042559 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040220 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 22:14:29.042559 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040223 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 22:14:29.042559 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040225 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 22:14:29.042559 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040228 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 22:14:29.042559 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040231 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 22:14:29.042559 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040233 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 22:14:29.042559 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040236 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 22:14:29.042559 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040239 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 22:14:29.042559 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040241 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 22:14:29.042559 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040244 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 22:14:29.043029 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040247 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 22:14:29.043029 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040249 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 22:14:29.043029 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040252 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 22:14:29.043029 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040254 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 22:14:29.043029 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040257 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 22:14:29.043029 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040260 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 22:14:29.043029 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040262 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 22:14:29.043029 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040264 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 22:14:29.043029 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040267 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 22:14:29.043029 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040270 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 22:14:29.043029 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040272 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 22:14:29.043029 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040275 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 22:14:29.043029 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040278 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 22:14:29.043029 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040281 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 22:14:29.043029 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040283 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 22:14:29.043029 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040286 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 22:14:29.043029 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040288 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 22:14:29.043029 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040291 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 22:14:29.043029 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040293 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 22:14:29.043029 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040295 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 22:14:29.043504 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040298 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 22:14:29.043504 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040300 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 22:14:29.043504 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040303 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 22:14:29.043504 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040305 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 22:14:29.043504 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040308 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 22:14:29.043504 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040310 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 22:14:29.043504 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040313 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 22:14:29.043504 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040316 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 22:14:29.043504 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040318 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 22:14:29.043504 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040321 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 22:14:29.043504 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040323 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 22:14:29.043504 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040326 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 22:14:29.043504 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040328 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 22:14:29.043504 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040331 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 22:14:29.043504 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040333 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 22:14:29.043504 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040335 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 22:14:29.043504 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040338 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 22:14:29.043504 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040340 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 22:14:29.043504 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040343 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 22:14:29.043504 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040345 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 22:14:29.043985 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040347 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 22:14:29.043985 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040350 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 22:14:29.043985 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040352 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 22:14:29.043985 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040355 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 22:14:29.043985 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040358 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 22:14:29.043985 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040360 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 22:14:29.043985 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040364 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 22:14:29.043985 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040373 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 22:14:29.043985 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040376 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 22:14:29.043985 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040378 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 22:14:29.043985 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040380 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 22:14:29.043985 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040383 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 22:14:29.043985 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040386 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 22:14:29.043985 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040388 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 22:14:29.043985 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040391 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 08 22:14:29.043985 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040393 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 22:14:29.043985 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040395 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 22:14:29.043985 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040399 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 22:14:29.043985 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040401 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 22:14:29.043985 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040404 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 22:14:29.044474 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040407 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 22:14:29.044474 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040411 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 22:14:29.044474 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040415 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 22:14:29.044474 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040418 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 22:14:29.044474 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040422 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 22:14:29.044474 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040425 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 22:14:29.044474 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040428 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 22:14:29.044474 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040430 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 22:14:29.044474 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040433 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 22:14:29.044474 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040435 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 22:14:29.044474 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040438 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 22:14:29.044474 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040441 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 22:14:29.044474 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040444 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 22:14:29.044474 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040448 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 22:14:29.044474 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040451 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 22:14:29.044474 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040455 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 22:14:29.044474 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.040458 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 22:14:29.044474 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040538 2563 flags.go:64] FLAG: --address="0.0.0.0" Jul 08 22:14:29.044474 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040551 2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 08 22:14:29.044474 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040561 2563 flags.go:64] FLAG: --anonymous-auth="true" Jul 08 22:14:29.044987 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040567 2563 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 08 22:14:29.044987 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040596 2563 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 08 22:14:29.044987 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040601 2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 08 22:14:29.044987 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040609 2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 08 22:14:29.044987 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040614 2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 08 22:14:29.044987 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040617 2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 08 22:14:29.044987 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040620 2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 08 22:14:29.044987 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040624 2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 08 22:14:29.044987 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040628 2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 08 22:14:29.044987 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040634 2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 08 22:14:29.044987 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040637 2563 flags.go:64] FLAG: --cgroup-root="" Jul 08 22:14:29.044987 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040640 2563 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 08 22:14:29.044987 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040643 2563 flags.go:64] FLAG: --client-ca-file="" Jul 08 22:14:29.044987 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040647 2563 flags.go:64] FLAG: --cloud-provider="external" Jul 08 22:14:29.044987 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040650 2563 flags.go:64] FLAG: --cluster-dns="[]" Jul 08 22:14:29.044987 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040655 2563 flags.go:64] FLAG: --cluster-domain="" Jul 08 22:14:29.044987 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040658 2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 08 22:14:29.044987 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040661 2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 08 22:14:29.044987 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040664 2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 08 22:14:29.044987 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040668 2563 flags.go:64] FLAG: --container-log-max-files="5" Jul 08 22:14:29.044987 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040672 2563 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 08 22:14:29.044987 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040675 2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 08 22:14:29.044987 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040678 2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 08 22:14:29.045522 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040682 2563 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 08 22:14:29.045522 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040685 2563 flags.go:64] FLAG: --contention-profiling="false" Jul 08 22:14:29.045522 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040688 2563 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 08 22:14:29.045522 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040691 2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 08 22:14:29.045522 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040695 2563 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 08 22:14:29.045522 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040698 2563 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 08 22:14:29.045522 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040702 2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 08 22:14:29.045522 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040705 2563 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 08 22:14:29.045522 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040708 2563 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 08 22:14:29.045522 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040710 2563 flags.go:64] FLAG: --enable-load-reader="false" Jul 08 22:14:29.045522 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040713 2563 flags.go:64] FLAG: --enable-server="true" Jul 08 22:14:29.045522 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040716 2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 08 22:14:29.045522 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040721 2563 flags.go:64] FLAG: --event-burst="100" Jul 08 22:14:29.045522 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040724 2563 flags.go:64] FLAG: --event-qps="50" Jul 08 22:14:29.045522 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040727 2563 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 08 22:14:29.045522 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040730 2563 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 08 22:14:29.045522 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040732 2563 flags.go:64] FLAG: --eviction-hard="" Jul 08 22:14:29.045522 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040736 2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 08 22:14:29.045522 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040741 2563 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 08 22:14:29.045522 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040744 2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 08 22:14:29.045522 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040747 2563 flags.go:64] FLAG: --eviction-soft="" Jul 08 22:14:29.045522 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040750 2563 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 08 22:14:29.045522 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040752 2563 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 08 22:14:29.045522 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040756 2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 08 22:14:29.045522 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040759 2563 flags.go:64] FLAG: --experimental-mounter-path="" Jul 08 22:14:29.046208 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040762 2563 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 08 22:14:29.046208 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040765 2563 flags.go:64] FLAG: --fail-swap-on="true" Jul 08 22:14:29.046208 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040768 2563 flags.go:64] FLAG: --feature-gates="" Jul 08 22:14:29.046208 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040771 2563 flags.go:64] FLAG: --file-check-frequency="20s" Jul 08 22:14:29.046208 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040774 2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 08 22:14:29.046208 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040778 2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 08 22:14:29.046208 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040781 2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 08 22:14:29.046208 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040784 2563 flags.go:64] FLAG: --healthz-port="10248" Jul 08 22:14:29.046208 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040787 2563 flags.go:64] FLAG: --help="false" Jul 08 22:14:29.046208 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040790 2563 flags.go:64] FLAG: --hostname-override="ip-10-0-133-162.ec2.internal" Jul 08 22:14:29.046208 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040793 2563 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 08 22:14:29.046208 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040796 2563 flags.go:64] FLAG: --http-check-frequency="20s" Jul 08 22:14:29.046208 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040799 2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 08 22:14:29.046208 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040802 2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 08 22:14:29.046208 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040806 2563 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 08 22:14:29.046208 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040809 2563 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 08 22:14:29.046208 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040811 2563 flags.go:64] FLAG: --image-service-endpoint="" Jul 08 22:14:29.046208 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040814 2563 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 08 22:14:29.046208 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040817 2563 flags.go:64] FLAG: --kube-api-burst="100" Jul 08 22:14:29.046208 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040820 2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 08 22:14:29.046208 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040823 2563 flags.go:64] FLAG: --kube-api-qps="50" Jul 08 22:14:29.046208 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040826 2563 flags.go:64] FLAG: --kube-reserved="" Jul 08 22:14:29.046208 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040829 2563 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 08 22:14:29.046208 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040832 2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 08 22:14:29.046791 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040835 2563 flags.go:64] FLAG: --kubelet-cgroups="" Jul 08 22:14:29.046791 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040839 2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 08 22:14:29.046791 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040842 2563 flags.go:64] FLAG: --lock-file="" Jul 08 22:14:29.046791 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040845 2563 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 08 22:14:29.046791 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040848 2563 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 08 22:14:29.046791 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040851 2563 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 08 22:14:29.046791 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040856 2563 flags.go:64] FLAG: --log-json-split-stream="false" Jul 08 22:14:29.046791 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040860 2563 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 08 22:14:29.046791 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040863 2563 flags.go:64] FLAG: --log-text-split-stream="false" Jul 08 22:14:29.046791 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040866 2563 flags.go:64] FLAG: --logging-format="text" Jul 08 22:14:29.046791 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040868 2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 08 22:14:29.046791 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040872 2563 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 08 22:14:29.046791 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040874 2563 flags.go:64] FLAG: --manifest-url="" Jul 08 22:14:29.046791 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040877 2563 flags.go:64] FLAG: --manifest-url-header="" Jul 08 22:14:29.046791 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040897 2563 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 08 22:14:29.046791 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040900 2563 flags.go:64] FLAG: --max-open-files="1000000" Jul 08 22:14:29.046791 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040905 2563 flags.go:64] FLAG: --max-pods="110" Jul 08 22:14:29.046791 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040907 2563 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 08 22:14:29.046791 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040910 2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 08 22:14:29.046791 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040913 2563 flags.go:64] FLAG: --memory-manager-policy="None" Jul 08 22:14:29.046791 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040916 2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 08 22:14:29.046791 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040919 2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 08 22:14:29.046791 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040922 2563 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 08 22:14:29.046791 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040925 2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 08 22:14:29.046791 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040932 2563 flags.go:64] FLAG: --node-status-max-images="50" Jul 08 22:14:29.047426 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040936 2563 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 08 22:14:29.047426 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040939 2563 flags.go:64] FLAG: --oom-score-adj="-999" Jul 08 22:14:29.047426 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040942 2563 flags.go:64] FLAG: --pod-cidr="" Jul 08 22:14:29.047426 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040945 2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 08 22:14:29.047426 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040951 2563 flags.go:64] FLAG: --pod-manifest-path="" Jul 08 22:14:29.047426 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040954 2563 flags.go:64] FLAG: --pod-max-pids="-1" Jul 08 22:14:29.047426 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040957 2563 flags.go:64] FLAG: --pods-per-core="0" Jul 08 22:14:29.047426 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040960 2563 flags.go:64] FLAG: --port="10250" Jul 08 22:14:29.047426 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040965 2563 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 08 22:14:29.047426 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040968 2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-062ef20bc52be9f1d" Jul 08 22:14:29.047426 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040971 2563 flags.go:64] FLAG: --qos-reserved="" Jul 08 22:14:29.047426 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040974 2563 flags.go:64] FLAG: --read-only-port="10255" Jul 08 22:14:29.047426 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040977 2563 flags.go:64] FLAG: --register-node="true" Jul 08 22:14:29.047426 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040980 2563 flags.go:64] FLAG: --register-with-taints="" Jul 08 22:14:29.047426 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040983 2563 flags.go:64] FLAG: --registry-burst="10" Jul 08 22:14:29.047426 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040987 2563 flags.go:64] FLAG: --registry-qps="5" Jul 08 22:14:29.047426 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040990 2563 flags.go:64] FLAG: --reserved-cpus="" Jul 08 22:14:29.047426 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040993 2563 flags.go:64] FLAG: --reserved-memory="" Jul 08 22:14:29.047426 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040996 2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 08 22:14:29.047426 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.040999 2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 08 22:14:29.047426 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.041002 2563 flags.go:64] FLAG: --rotate-certificates="false" Jul 08 22:14:29.047426 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.041005 2563 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 08 22:14:29.047426 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.041008 2563 flags.go:64] FLAG: --runonce="false" Jul 08 22:14:29.047426 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.041011 2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 08 22:14:29.047426 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.041014 2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 08 22:14:29.048033 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.041017 2563 flags.go:64] FLAG: --seccomp-default="false" Jul 08 22:14:29.048033 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.041020 2563 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 08 22:14:29.048033 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.041023 2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 08 22:14:29.048033 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.041026 2563 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 08 22:14:29.048033 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.041029 2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 08 22:14:29.048033 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.041032 2563 flags.go:64] FLAG: --storage-driver-password="root" Jul 08 22:14:29.048033 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.041035 2563 flags.go:64] FLAG: --storage-driver-secure="false" Jul 08 22:14:29.048033 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.041038 2563 flags.go:64] FLAG: --storage-driver-table="stats" Jul 08 22:14:29.048033 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.041041 2563 flags.go:64] FLAG: --storage-driver-user="root" Jul 08 22:14:29.048033 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.041043 2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 08 22:14:29.048033 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.041046 2563 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 08 22:14:29.048033 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.041049 2563 flags.go:64] FLAG: --system-cgroups="" Jul 08 22:14:29.048033 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.041052 2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 08 22:14:29.048033 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.041057 2563 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 08 22:14:29.048033 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.041060 2563 flags.go:64] FLAG: --tls-cert-file="" Jul 08 22:14:29.048033 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.041064 2563 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 08 22:14:29.048033 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.041068 2563 flags.go:64] FLAG: --tls-min-version="" Jul 08 22:14:29.048033 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.041071 2563 flags.go:64] FLAG: --tls-private-key-file="" Jul 08 22:14:29.048033 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.041074 2563 flags.go:64] FLAG: --topology-manager-policy="none" Jul 08 22:14:29.048033 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.041077 2563 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 08 22:14:29.048033 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.041080 2563 flags.go:64] FLAG: --topology-manager-scope="container" Jul 08 22:14:29.048033 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.041082 2563 flags.go:64] FLAG: --v="2" Jul 08 22:14:29.048033 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.041086 2563 flags.go:64] FLAG: --version="false" Jul 08 22:14:29.048033 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.041091 2563 flags.go:64] FLAG: --vmodule="" Jul 08 22:14:29.048033 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.041095 2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 08 22:14:29.048690 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.041098 2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 08 22:14:29.048690 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041221 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 22:14:29.048690 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041224 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 22:14:29.048690 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041227 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 22:14:29.048690 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041232 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 22:14:29.048690 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041235 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 22:14:29.048690 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041237 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 22:14:29.048690 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041240 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 22:14:29.048690 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041243 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 22:14:29.048690 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041246 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 22:14:29.048690 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041249 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 22:14:29.048690 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041253 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 22:14:29.048690 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041257 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 22:14:29.048690 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041260 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 22:14:29.048690 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041263 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 22:14:29.048690 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041266 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 22:14:29.048690 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041269 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 22:14:29.048690 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041272 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 22:14:29.048690 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041275 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 22:14:29.048690 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041277 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 22:14:29.049191 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041280 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 22:14:29.049191 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041282 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 22:14:29.049191 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041286 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 22:14:29.049191 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041289 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 22:14:29.049191 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041292 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 22:14:29.049191 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041294 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 22:14:29.049191 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041298 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 22:14:29.049191 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041300 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 22:14:29.049191 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041303 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 22:14:29.049191 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041305 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 22:14:29.049191 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041308 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 22:14:29.049191 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041319 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 22:14:29.049191 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041322 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 22:14:29.049191 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041325 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 22:14:29.049191 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041327 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 22:14:29.049191 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041330 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 22:14:29.049191 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041334 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 22:14:29.049191 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041337 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 22:14:29.049191 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041339 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 22:14:29.049653 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041342 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 22:14:29.049653 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041346 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 22:14:29.049653 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041349 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 22:14:29.049653 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041352 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 22:14:29.049653 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041355 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 22:14:29.049653 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041358 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 22:14:29.049653 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041361 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 22:14:29.049653 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041363 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 22:14:29.049653 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041366 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 22:14:29.049653 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041369 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 08 22:14:29.049653 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041371 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 22:14:29.049653 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041374 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 22:14:29.049653 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041376 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 22:14:29.049653 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041379 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 22:14:29.049653 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041381 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 22:14:29.049653 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041385 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 22:14:29.049653 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041388 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 22:14:29.049653 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041390 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 22:14:29.049653 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041393 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 22:14:29.049653 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041395 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 22:14:29.050161 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041398 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 22:14:29.050161 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041401 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 22:14:29.050161 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041403 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 22:14:29.050161 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041406 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 22:14:29.050161 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041410 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 22:14:29.050161 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041414 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 22:14:29.050161 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041418 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 22:14:29.050161 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041422 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 22:14:29.050161 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041426 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 22:14:29.050161 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041435 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 22:14:29.050161 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041438 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 22:14:29.050161 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041441 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 22:14:29.050161 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041443 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 22:14:29.050161 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041446 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 22:14:29.050161 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041448 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 22:14:29.050161 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041451 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 22:14:29.050161 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041454 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 22:14:29.050161 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041456 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 22:14:29.050161 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041459 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 22:14:29.050161 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041462 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 22:14:29.050683 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041464 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 22:14:29.050683 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041467 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 22:14:29.050683 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041470 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 22:14:29.050683 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041472 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 22:14:29.050683 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041476 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 22:14:29.050683 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041478 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 22:14:29.050683 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041481 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 22:14:29.050683 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041484 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 22:14:29.050683 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041487 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 22:14:29.050683 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041489 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 22:14:29.050683 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041493 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 22:14:29.050683 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041498 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 22:14:29.050683 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041501 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 22:14:29.050683 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041503 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 22:14:29.050683 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041506 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 22:14:29.050683 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041508 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 22:14:29.050683 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041511 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 22:14:29.050683 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041513 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 22:14:29.050683 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041517 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 22:14:29.051199 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041519 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 22:14:29.051199 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.041527 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 22:14:29.051199 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.042776 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 22:14:29.051199 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.050133 2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 08 22:14:29.051199 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.050153 2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 08 22:14:29.051199 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.051166 2563 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 08 22:14:29.051199 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.051175 2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 08 22:14:29.051380 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051244 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 22:14:29.051380 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051250 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 22:14:29.051380 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051253 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 22:14:29.051380 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051257 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 22:14:29.051380 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051260 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 22:14:29.051380 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051263 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 22:14:29.051380 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051266 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 22:14:29.051380 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051269 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 22:14:29.051380 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051272 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 22:14:29.051380 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051275 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 22:14:29.051380 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051278 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 22:14:29.051380 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051281 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 22:14:29.051380 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051284 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 22:14:29.051380 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051287 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 22:14:29.051380 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051290 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 22:14:29.051380 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051293 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 22:14:29.051380 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051296 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 22:14:29.051380 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051299 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 22:14:29.051380 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051302 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 22:14:29.051380 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051305 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 22:14:29.051864 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051309 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 22:14:29.051864 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051313 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 22:14:29.051864 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051316 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 22:14:29.051864 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051319 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 22:14:29.051864 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051323 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 22:14:29.051864 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051326 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 22:14:29.051864 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051329 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 22:14:29.051864 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051332 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 22:14:29.051864 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051334 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 22:14:29.051864 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051338 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 22:14:29.051864 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051341 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 22:14:29.051864 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051344 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 22:14:29.051864 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051347 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 22:14:29.051864 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051350 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 22:14:29.051864 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051353 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 22:14:29.051864 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051356 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 22:14:29.051864 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051359 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 22:14:29.051864 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051361 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 22:14:29.051864 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051365 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 22:14:29.052335 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051369 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 22:14:29.052335 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051371 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 22:14:29.052335 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051375 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 22:14:29.052335 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051378 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 22:14:29.052335 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051381 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 22:14:29.052335 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051384 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 22:14:29.052335 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051386 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 22:14:29.052335 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051389 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 22:14:29.052335 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051392 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 22:14:29.052335 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051396 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 22:14:29.052335 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051398 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 22:14:29.052335 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051401 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 22:14:29.052335 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051404 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 22:14:29.052335 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051406 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 22:14:29.052335 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051409 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 22:14:29.052335 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051411 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 22:14:29.052335 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051414 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 22:14:29.052335 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051417 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 22:14:29.052335 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051420 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 22:14:29.052800 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051423 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 22:14:29.052800 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051426 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 22:14:29.052800 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051428 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 22:14:29.052800 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051431 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 22:14:29.052800 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051435 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 22:14:29.052800 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051437 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 22:14:29.052800 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051440 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 22:14:29.052800 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051445 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 22:14:29.052800 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051448 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 22:14:29.052800 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051450 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 22:14:29.052800 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051453 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 22:14:29.052800 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051456 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 22:14:29.052800 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051467 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 22:14:29.052800 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051470 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 22:14:29.052800 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051473 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 22:14:29.052800 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051475 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 22:14:29.052800 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051478 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 22:14:29.052800 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051480 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 22:14:29.052800 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051483 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 22:14:29.052800 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051486 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 22:14:29.053297 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051488 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 22:14:29.053297 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051491 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 22:14:29.053297 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051493 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 22:14:29.053297 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051496 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 22:14:29.053297 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051498 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 22:14:29.053297 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051501 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 22:14:29.053297 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051503 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 22:14:29.053297 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051506 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 22:14:29.053297 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051508 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 22:14:29.053297 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051511 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 22:14:29.053297 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051514 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 22:14:29.053297 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051517 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 22:14:29.053297 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051519 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 22:14:29.053297 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051522 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 22:14:29.053297 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051524 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 22:14:29.053297 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051527 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 22:14:29.053297 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051530 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 22:14:29.053297 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051533 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 22:14:29.053297 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051536 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 22:14:29.053297 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051539 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 22:14:29.053775 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051541 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 08 22:14:29.053775 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.051547 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 22:14:29.053775 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051669 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 22:14:29.053775 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051675 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 22:14:29.053775 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051677 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 22:14:29.053775 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051681 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 22:14:29.053775 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051683 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 22:14:29.053775 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051686 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 22:14:29.053775 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051688 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 22:14:29.053775 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051691 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 22:14:29.053775 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051694 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 22:14:29.053775 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051696 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 22:14:29.053775 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051699 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 22:14:29.053775 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051701 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 22:14:29.053775 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051704 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 22:14:29.054166 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051706 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 22:14:29.054166 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051709 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 22:14:29.054166 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051711 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 22:14:29.054166 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051714 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 22:14:29.054166 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051716 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 22:14:29.054166 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051719 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 22:14:29.054166 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051721 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 22:14:29.054166 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051724 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 22:14:29.054166 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051727 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 22:14:29.054166 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051730 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 22:14:29.054166 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051733 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 08 22:14:29.054166 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051735 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 22:14:29.054166 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051738 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 22:14:29.054166 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051740 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 22:14:29.054166 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051743 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 22:14:29.054166 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051746 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 22:14:29.054166 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051749 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 22:14:29.054166 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051752 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 22:14:29.054166 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051755 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 22:14:29.054166 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051757 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 22:14:29.054669 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051760 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 22:14:29.054669 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051763 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 22:14:29.054669 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051766 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 22:14:29.054669 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051768 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 22:14:29.054669 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051771 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 22:14:29.054669 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051773 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 22:14:29.054669 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051776 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 22:14:29.054669 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051778 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 22:14:29.054669 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051781 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 22:14:29.054669 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051783 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 22:14:29.054669 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051786 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 22:14:29.054669 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051790 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 22:14:29.054669 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051794 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 22:14:29.054669 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051797 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 22:14:29.054669 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051799 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 22:14:29.054669 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051803 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 22:14:29.054669 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051805 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 22:14:29.054669 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051809 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 22:14:29.054669 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051811 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 22:14:29.055170 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051815 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 22:14:29.055170 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051817 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 22:14:29.055170 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051820 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 22:14:29.055170 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051823 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 22:14:29.055170 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051825 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 22:14:29.055170 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051828 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 22:14:29.055170 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051830 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 22:14:29.055170 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051833 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 22:14:29.055170 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051835 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 22:14:29.055170 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051839 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 22:14:29.055170 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051842 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 22:14:29.055170 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051845 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 22:14:29.055170 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051847 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 22:14:29.055170 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051850 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 22:14:29.055170 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051852 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 22:14:29.055170 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051855 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 22:14:29.055170 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051857 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 22:14:29.055170 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051860 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 22:14:29.055170 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051862 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 22:14:29.055170 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051865 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 22:14:29.055657 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051867 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 22:14:29.055657 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051870 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 22:14:29.055657 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051873 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 22:14:29.055657 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051876 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 22:14:29.055657 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051879 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 22:14:29.055657 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051892 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 22:14:29.055657 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051895 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 22:14:29.055657 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051898 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 22:14:29.055657 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051900 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 22:14:29.055657 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051903 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 22:14:29.055657 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051906 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 22:14:29.055657 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051908 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 22:14:29.055657 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051911 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 22:14:29.055657 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051914 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 22:14:29.055657 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051916 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 22:14:29.055657 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051919 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 22:14:29.055657 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051922 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 22:14:29.055657 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051926 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 22:14:29.055657 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051929 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 22:14:29.055657 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051932 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 22:14:29.056259 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051935 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 22:14:29.056259 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051938 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 22:14:29.056259 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051942 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 22:14:29.056259 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051945 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 22:14:29.056259 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051948 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 22:14:29.056259 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051951 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 22:14:29.056259 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:29.051954 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 22:14:29.056259 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.051959 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 22:14:29.056259 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.052112 2563 server.go:962] "Client rotation is on, will bootstrap in background" Jul 08 22:14:29.056259 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.054448 2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 08 22:14:29.056259 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.055936 2563 server.go:1019] "Starting client certificate rotation" Jul 08 22:14:29.056259 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.056055 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 22:14:29.056259 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.056106 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 22:14:29.060371 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.060349 2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 22:14:29.064599 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.064547 2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 22:14:29.076322 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.076292 2563 log.go:25] "Validated CRI v1 runtime API" Jul 08 22:14:29.078860 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.078846 2563 log.go:25] "Validated CRI v1 image API" Jul 08 22:14:29.079722 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.079703 2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 08 22:14:29.083355 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.083328 2563 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 7e45d9eb-c79c-4bc2-bfe8-eb03a00ab5fc:/dev/nvme0n1p3 e88bd338-1f81-4645-ac19-e55a0c177096:/dev/nvme0n1p4] Jul 08 22:14:29.083435 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.083350 2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 08 22:14:29.086164 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.086145 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 22:14:29.089085 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.088967 2563 manager.go:216] Machine: {Timestamp:2026-07-08 22:14:29.087468374 +0000 UTC m=+0.355488392 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100313 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec230a4c50a3f9bd88b887a091e2d9e6 SystemUUID:ec230a4c-50a3-f9bd-88b8-87a091e2d9e6 BootID:2c8d7ea4-e238-4396-ab79-f75880cdab9d Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:59:f9:c3:34:5f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:59:f9:c3:34:5f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5e:c0:9e:c0:2b:2a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 08 22:14:29.089085 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.089084 2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 08 22:14:29.089210 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.089175 2563 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 08 22:14:29.092707 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.092678 2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 08 22:14:29.092856 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.092710 2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-162.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 08 22:14:29.094039 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.094027 2563 topology_manager.go:138] "Creating topology manager with none policy" Jul 08 22:14:29.094079 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.094040 2563 container_manager_linux.go:309] "Creating device plugin manager" Jul 08 22:14:29.094079 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.094049 2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 22:14:29.094079 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.094064 2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 22:14:29.096342 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.096330 2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 08 22:14:29.099947 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.099933 2563 state_mem.go:36] "Initialized new in-memory state store" Jul 08 22:14:29.100084 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.100074 2563 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 08 22:14:29.100114 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.100106 2563 kubelet.go:486] "Attempting to sync node with API server" Jul 08 22:14:29.100149 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.100119 2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 08 22:14:29.100149 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.100146 2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 08 22:14:29.100205 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.100159 2563 kubelet.go:398] "Adding apiserver pod source" Jul 08 22:14:29.100205 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.100169 2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 08 22:14:29.100686 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.100669 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 22:14:29.100686 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.100689 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 22:14:29.103479 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.103452 2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 08 22:14:29.105705 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.105686 2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 08 22:14:29.105761 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.105722 2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 08 22:14:29.107380 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.107368 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 08 22:14:29.107415 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.107386 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 08 22:14:29.107415 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.107394 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 08 22:14:29.107415 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.107402 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 08 22:14:29.107415 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.107410 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 08 22:14:29.107415 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.107416 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 08 22:14:29.107546 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.107424 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 08 22:14:29.107546 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.107430 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 08 22:14:29.107546 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.107441 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 08 22:14:29.107546 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.107447 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 08 22:14:29.107546 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.107456 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 08 22:14:29.107546 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.107461 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 08 22:14:29.108239 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.108222 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 08 22:14:29.108239 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.108238 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 08 22:14:29.108406 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.108232 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kb2ck" Jul 08 22:14:29.110244 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:29.110218 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 08 22:14:29.110314 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:29.110218 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-162.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 08 22:14:29.110314 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.110225 2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-162.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 22:14:29.111834 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.111822 2563 server.go:1268] "Started kubelet" Jul 08 22:14:29.111951 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.111899 2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 08 22:14:29.112225 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.111955 2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 08 22:14:29.112757 ip-10-0-133-162 systemd[1]: Started Kubernetes Kubelet. Jul 08 22:14:29.115427 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.115363 2563 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 08 22:14:29.115953 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.115931 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kb2ck" Jul 08 22:14:29.117012 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.116993 2563 server.go:310] "Adding debug handlers to kubelet server" Jul 08 22:14:29.118195 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.118162 2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 08 22:14:29.118557 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:29.118536 2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 08 22:14:29.121782 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:29.120828 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-162.ec2.internal.18c0703f2ef5482e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-162.ec2.internal,UID:ip-10-0-133-162.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-162.ec2.internal,},FirstTimestamp:2026-07-08 22:14:29.11179371 +0000 UTC m=+0.379813729,LastTimestamp:2026-07-08 22:14:29.11179371 +0000 UTC m=+0.379813729,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-162.ec2.internal,}" Jul 08 22:14:29.125578 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.125562 2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 08 22:14:29.125657 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.125578 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 08 22:14:29.125657 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:29.125641 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-162.ec2.internal\" not found" Jul 08 22:14:29.126349 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.126334 2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 08 22:14:29.126736 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.126491 2563 factory.go:55] Registering systemd factory Jul 08 22:14:29.126736 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.126530 2563 factory.go:223] Registration of the systemd container factory successfully Jul 08 22:14:29.126824 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.126744 2563 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 08 22:14:29.126824 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.126760 2563 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 08 22:14:29.126824 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.126763 2563 factory.go:153] Registering CRI-O factory Jul 08 22:14:29.126824 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.126780 2563 factory.go:223] Registration of the crio container factory successfully Jul 08 22:14:29.127015 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.126836 2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 08 22:14:29.127015 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.126862 2563 factory.go:103] Registering Raw factory Jul 08 22:14:29.127015 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.126877 2563 manager.go:1196] Started watching for new ooms in manager Jul 08 22:14:29.127015 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.126915 2563 reconstruct.go:97] "Volume reconstruction finished" Jul 08 22:14:29.127015 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.126923 2563 reconciler.go:29] "Reconciler: start to sync state" Jul 08 22:14:29.127636 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.127617 2563 manager.go:349] Starting recovery of all containers Jul 08 22:14:29.136545 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.136521 2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 22:14:29.139332 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.139315 2563 manager.go:354] Recovery completed Jul 08 22:14:29.139750 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:29.139711 2563 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-133-162.ec2.internal\" not found" node="ip-10-0-133-162.ec2.internal" Jul 08 22:14:29.143462 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.143448 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 22:14:29.146785 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.146767 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-162.ec2.internal" event="NodeHasSufficientMemory" Jul 08 22:14:29.146879 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.146797 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-162.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 22:14:29.146879 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.146807 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-162.ec2.internal" event="NodeHasSufficientPID" Jul 08 22:14:29.147419 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.147405 2563 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 08 22:14:29.147419 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.147413 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 22:14:29.147518 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.147417 2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 08 22:14:29.147567 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.147521 2563 state_mem.go:36] "Initialized new in-memory state store" Jul 08 22:14:29.148168 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.148153 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-162.ec2.internal" event="NodeHasSufficientMemory" Jul 08 22:14:29.148240 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.148183 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-162.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 22:14:29.148240 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.148195 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-162.ec2.internal" event="NodeHasSufficientPID" Jul 08 22:14:29.148949 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.148938 2563 policy_none.go:49] "None policy: Start" Jul 08 22:14:29.148984 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.148953 2563 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 08 22:14:29.148984 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.148964 2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 08 22:14:29.150166 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.150155 2563 policy_none.go:47] "Start" Jul 08 22:14:29.188639 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.188622 2563 manager.go:337] "Starting Device Plugin manager" Jul 08 22:14:29.188752 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:29.188661 2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 08 22:14:29.188752 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.188672 2563 server.go:87] "Starting device plugin registration server" Jul 08 22:14:29.189007 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.188993 2563 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 08 22:14:29.189114 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.189011 2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 08 22:14:29.189153 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.189144 2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 08 22:14:29.189290 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.189224 2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 08 22:14:29.189290 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.189237 2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 08 22:14:29.189757 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:29.189735 2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 08 22:14:29.189851 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:29.189793 2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-162.ec2.internal\" not found" Jul 08 22:14:29.289490 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.289406 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 22:14:29.290802 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.290592 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-162.ec2.internal" event="NodeHasSufficientMemory" Jul 08 22:14:29.290802 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.290624 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-162.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 22:14:29.290802 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.290635 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-162.ec2.internal" event="NodeHasSufficientPID" Jul 08 22:14:29.290802 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.290663 2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-162.ec2.internal" Jul 08 22:14:29.298515 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.298498 2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-162.ec2.internal" Jul 08 22:14:29.298569 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:29.298523 2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-162.ec2.internal\": node \"ip-10-0-133-162.ec2.internal\" not found" Jul 08 22:14:29.310659 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.310614 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 08 22:14:29.311982 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.311957 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 08 22:14:29.312123 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.312002 2563 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 08 22:14:29.312123 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.312028 2563 kubelet.go:2448] "Starting kubelet main sync loop" Jul 08 22:14:29.312123 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:29.312070 2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 08 22:14:29.314300 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.314278 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 22:14:29.315612 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:29.315594 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-162.ec2.internal\" not found" Jul 08 22:14:29.412414 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.412361 2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-162.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-162.ec2.internal"] Jul 08 22:14:29.412553 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.412463 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 22:14:29.413509 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.413479 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-162.ec2.internal" event="NodeHasSufficientMemory" Jul 08 22:14:29.413579 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.413511 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-162.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 22:14:29.413579 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.413521 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-162.ec2.internal" event="NodeHasSufficientPID" Jul 08 22:14:29.415097 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.415085 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 22:14:29.415213 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.415196 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-162.ec2.internal" Jul 08 22:14:29.415273 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.415229 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 22:14:29.415719 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:29.415698 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-162.ec2.internal\" not found" Jul 08 22:14:29.415876 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.415861 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-162.ec2.internal" event="NodeHasSufficientMemory" Jul 08 22:14:29.415940 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.415909 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-162.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 22:14:29.415940 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.415925 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-162.ec2.internal" event="NodeHasSufficientPID" Jul 08 22:14:29.415940 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.415925 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-162.ec2.internal" event="NodeHasSufficientMemory" Jul 08 22:14:29.416035 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.415944 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-162.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 22:14:29.416035 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.415953 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-162.ec2.internal" event="NodeHasSufficientPID" Jul 08 22:14:29.417161 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.417142 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-162.ec2.internal" Jul 08 22:14:29.417253 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.417173 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 22:14:29.417978 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.417965 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-162.ec2.internal" event="NodeHasSufficientMemory" Jul 08 22:14:29.418066 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.417992 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-162.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 22:14:29.418066 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.418006 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-162.ec2.internal" event="NodeHasSufficientPID" Jul 08 22:14:29.429130 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.429105 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/55e574bf7ca2ba660429fe3c53ea934c-config\") pod \"kube-apiserver-proxy-ip-10-0-133-162.ec2.internal\" (UID: \"55e574bf7ca2ba660429fe3c53ea934c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-162.ec2.internal" Jul 08 22:14:29.429200 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.429134 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b96734b96bcdda2fdee4b1945ad78b59-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-162.ec2.internal\" (UID: \"b96734b96bcdda2fdee4b1945ad78b59\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-162.ec2.internal" Jul 08 22:14:29.429200 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.429153 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b96734b96bcdda2fdee4b1945ad78b59-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-162.ec2.internal\" (UID: \"b96734b96bcdda2fdee4b1945ad78b59\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-162.ec2.internal" Jul 08 22:14:29.440816 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:29.440794 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-162.ec2.internal\" not found" node="ip-10-0-133-162.ec2.internal" Jul 08 22:14:29.445397 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:29.445379 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-162.ec2.internal\" not found" node="ip-10-0-133-162.ec2.internal" Jul 08 22:14:29.516143 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:29.516104 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-162.ec2.internal\" not found" Jul 08 22:14:29.529535 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.529512 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/55e574bf7ca2ba660429fe3c53ea934c-config\") pod \"kube-apiserver-proxy-ip-10-0-133-162.ec2.internal\" (UID: \"55e574bf7ca2ba660429fe3c53ea934c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-162.ec2.internal" Jul 08 22:14:29.529645 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.529541 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b96734b96bcdda2fdee4b1945ad78b59-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-162.ec2.internal\" (UID: \"b96734b96bcdda2fdee4b1945ad78b59\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-162.ec2.internal" Jul 08 22:14:29.529645 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.529558 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b96734b96bcdda2fdee4b1945ad78b59-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-162.ec2.internal\" (UID: \"b96734b96bcdda2fdee4b1945ad78b59\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-162.ec2.internal" Jul 08 22:14:29.529645 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.529613 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b96734b96bcdda2fdee4b1945ad78b59-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-162.ec2.internal\" (UID: \"b96734b96bcdda2fdee4b1945ad78b59\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-162.ec2.internal" Jul 08 22:14:29.529751 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.529642 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b96734b96bcdda2fdee4b1945ad78b59-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-162.ec2.internal\" (UID: \"b96734b96bcdda2fdee4b1945ad78b59\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-162.ec2.internal" Jul 08 22:14:29.529751 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.529644 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/55e574bf7ca2ba660429fe3c53ea934c-config\") pod \"kube-apiserver-proxy-ip-10-0-133-162.ec2.internal\" (UID: \"55e574bf7ca2ba660429fe3c53ea934c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-162.ec2.internal" Jul 08 22:14:29.617047 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:29.616971 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-162.ec2.internal\" not found" Jul 08 22:14:29.717548 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:29.717517 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-162.ec2.internal\" not found" Jul 08 22:14:29.742969 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.742948 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-162.ec2.internal" Jul 08 22:14:29.749180 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:29.749156 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-162.ec2.internal" Jul 08 22:14:29.817964 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:29.817925 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-162.ec2.internal\" not found" Jul 08 22:14:29.918588 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:29.918505 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-162.ec2.internal\" not found" Jul 08 22:14:30.019077 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:30.019033 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-162.ec2.internal\" not found" Jul 08 22:14:30.055698 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:30.055671 2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 08 22:14:30.056183 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:30.055826 2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 22:14:30.056183 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:30.055858 2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 22:14:30.119209 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:30.119179 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-162.ec2.internal\" not found" Jul 08 22:14:30.122679 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:30.122650 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 22:09:29 +0000 UTC" deadline="2027-12-31 22:08:05.204336394 +0000 UTC" Jul 08 22:14:30.122679 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:30.122677 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12983h53m35.081662071s" Jul 08 22:14:30.219748 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:30.219677 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-162.ec2.internal\" not found" Jul 08 22:14:30.245250 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:30.245220 2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 22:14:30.320506 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:30.320472 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-162.ec2.internal\" not found" Jul 08 22:14:30.347692 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:30.347658 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb96734b96bcdda2fdee4b1945ad78b59.slice/crio-897fab4aee666fdb6acb7191a20afd09a685c79d84a58aa25ffebfc6fe1a36fb WatchSource:0}: Error finding container 897fab4aee666fdb6acb7191a20afd09a685c79d84a58aa25ffebfc6fe1a36fb: Status 404 returned error can't find the container with id 897fab4aee666fdb6acb7191a20afd09a685c79d84a58aa25ffebfc6fe1a36fb Jul 08 22:14:30.348164 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:30.348135 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55e574bf7ca2ba660429fe3c53ea934c.slice/crio-feb29f034f55c18062e4bd1224f3cddb5047afb363ebd3c827436f8de84537be WatchSource:0}: Error finding container feb29f034f55c18062e4bd1224f3cddb5047afb363ebd3c827436f8de84537be: Status 404 returned error can't find the container with id feb29f034f55c18062e4bd1224f3cddb5047afb363ebd3c827436f8de84537be Jul 08 22:14:30.353232 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:30.353214 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 22:14:30.421391 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:30.421347 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-162.ec2.internal\" not found" Jul 08 22:14:30.422103 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:30.422086 2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 22:14:30.426566 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:30.426547 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-162.ec2.internal" Jul 08 22:14:30.442277 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:30.442254 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 22:14:30.443239 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:30.443223 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-162.ec2.internal" Jul 08 22:14:30.459024 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:30.459001 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 22:14:30.860580 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:30.860534 2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 22:14:30.912324 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:30.912268 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 22:14:31.101279 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.101248 2563 apiserver.go:52] "Watching apiserver" Jul 08 22:14:31.108869 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.108843 2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 08 22:14:31.109345 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.109312 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-jhpj6","openshift-network-diagnostics/network-check-target-98x2n","openshift-network-operator/iptables-alerter-45nvq","openshift-ovn-kubernetes/ovnkube-node-bltvn","kube-system/global-pull-secret-syncer-fffds","kube-system/kube-apiserver-proxy-ip-10-0-133-162.ec2.internal","openshift-dns/node-resolver-mzzbg","openshift-image-registry/node-ca-4rvb8","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-162.ec2.internal","openshift-multus/multus-8fgq2","kube-system/konnectivity-agent-p4pz6","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfvvt","openshift-cluster-node-tuning-operator/tuned-bj85s","openshift-multus/multus-additional-cni-plugins-fbm5c"] Jul 08 22:14:31.111084 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.111029 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mzzbg" Jul 08 22:14:31.113242 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.113220 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.113614 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.113596 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 08 22:14:31.114924 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.114265 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4dz5x\"" Jul 08 22:14:31.114924 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.113604 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-45nvq" Jul 08 22:14:31.119910 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.116076 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-r7wbp\"" Jul 08 22:14:31.119910 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.116337 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 08 22:14:31.119910 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.116823 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 08 22:14:31.119910 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.117074 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 08 22:14:31.119910 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.117749 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 08 22:14:31.119910 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.118002 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-ftrh7\"" Jul 08 22:14:31.119910 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.118171 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 08 22:14:31.119910 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.118278 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 08 22:14:31.122220 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.121939 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.123876 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.123545 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:14:31.123876 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:31.123616 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fffds" podUID="4a354dc2-56c0-4280-96d1-b560d70be2b2" Jul 08 22:14:31.124495 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.124474 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 08 22:14:31.127496 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.125393 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 08 22:14:31.127496 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.125603 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 08 22:14:31.127496 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.125307 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jdggx\"" Jul 08 22:14:31.127496 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.125836 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 08 22:14:31.127496 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.126112 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fbm5c" Jul 08 22:14:31.127496 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.126137 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 08 22:14:31.127496 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.126320 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 08 22:14:31.127496 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.126789 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 08 22:14:31.127994 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.127923 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4rvb8" Jul 08 22:14:31.130124 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.130103 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8wb8n\"" Jul 08 22:14:31.130298 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.130279 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 08 22:14:31.130373 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.130318 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 08 22:14:31.130471 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.130454 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 08 22:14:31.130549 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.130112 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 08 22:14:31.130811 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.130700 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:14:31.130811 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:31.130771 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhpj6" podUID="7c5c6b58-e07d-4d72-af33-ebe169402d4e" Jul 08 22:14:31.131132 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.131112 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:14:31.131214 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:31.131186 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98x2n" podUID="6a151365-c622-4361-862f-ed202f3045f7" Jul 08 22:14:31.131617 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.131593 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 08 22:14:31.131700 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.131679 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 08 22:14:31.131825 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.131595 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-xn5fh\"" Jul 08 22:14:31.131986 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.131970 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 08 22:14:31.132088 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.132073 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 08 22:14:31.132709 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.132692 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.134280 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.134113 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-p4pz6" Jul 08 22:14:31.135193 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.135079 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-cjnkb\"" Jul 08 22:14:31.135193 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.135093 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 08 22:14:31.135535 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.135516 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfvvt" Jul 08 22:14:31.137317 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.137139 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 08 22:14:31.137317 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.137264 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hrdmj\"" Jul 08 22:14:31.137317 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.137271 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 08 22:14:31.137948 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.137731 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 08 22:14:31.138080 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.138062 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-zb4dk\"" Jul 08 22:14:31.138259 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.138239 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 08 22:14:31.138349 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.138297 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 08 22:14:31.139044 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.139020 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-run-ovn\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.139131 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.139058 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4928f3d7-5f78-4844-9103-cb23d99e9883-ovnkube-script-lib\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.139131 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.139087 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e722d997-2e47-40ac-bb5c-48e25dde7cfd-cni-binary-copy\") pod \"multus-additional-cni-plugins-fbm5c\" (UID: \"e722d997-2e47-40ac-bb5c-48e25dde7cfd\") " pod="openshift-multus/multus-additional-cni-plugins-fbm5c" Jul 08 22:14:31.139131 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.139112 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/15388137-d252-4a28-996c-3f0ca3236e35-sys\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.139282 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.139181 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-etc-openvswitch\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.139282 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.139232 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e722d997-2e47-40ac-bb5c-48e25dde7cfd-system-cni-dir\") pod \"multus-additional-cni-plugins-fbm5c\" (UID: \"e722d997-2e47-40ac-bb5c-48e25dde7cfd\") " pod="openshift-multus/multus-additional-cni-plugins-fbm5c" Jul 08 22:14:31.139282 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.139265 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmgj9\" (UniqueName: \"kubernetes.io/projected/b0e80baf-e49e-4b91-ab27-dbc8dc3c2dc4-kube-api-access-kmgj9\") pod \"node-ca-4rvb8\" (UID: \"b0e80baf-e49e-4b91-ab27-dbc8dc3c2dc4\") " pod="openshift-image-registry/node-ca-4rvb8" Jul 08 22:14:31.139423 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.139290 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e722d997-2e47-40ac-bb5c-48e25dde7cfd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fbm5c\" (UID: \"e722d997-2e47-40ac-bb5c-48e25dde7cfd\") " pod="openshift-multus/multus-additional-cni-plugins-fbm5c" Jul 08 22:14:31.139423 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.139313 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-systemd-units\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.139423 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.139341 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/15388137-d252-4a28-996c-3f0ca3236e35-etc-modprobe-d\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.139423 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.139370 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/15388137-d252-4a28-996c-3f0ca3236e35-etc-kubernetes\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.139423 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.139392 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.139423 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.139409 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4a354dc2-56c0-4280-96d1-b560d70be2b2-kubelet-config\") pod \"global-pull-secret-syncer-fffds\" (UID: \"4a354dc2-56c0-4280-96d1-b560d70be2b2\") " pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:14:31.139423 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.139424 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/15388137-d252-4a28-996c-3f0ca3236e35-run\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.139739 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.139458 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/15388137-d252-4a28-996c-3f0ca3236e35-var-lib-kubelet\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.139739 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.139501 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/15388137-d252-4a28-996c-3f0ca3236e35-host\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.139739 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.139527 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-node-log\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.139739 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.139551 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4928f3d7-5f78-4844-9103-cb23d99e9883-env-overrides\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.139739 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.139587 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4a354dc2-56c0-4280-96d1-b560d70be2b2-dbus\") pod \"global-pull-secret-syncer-fffds\" (UID: \"4a354dc2-56c0-4280-96d1-b560d70be2b2\") " pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:14:31.139739 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.139612 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a354dc2-56c0-4280-96d1-b560d70be2b2-original-pull-secret\") pod \"global-pull-secret-syncer-fffds\" (UID: \"4a354dc2-56c0-4280-96d1-b560d70be2b2\") " pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:14:31.139739 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.139708 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-var-lib-openvswitch\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.140085 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.139744 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-run-openvswitch\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.140085 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.139825 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-host-run-netns\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.140085 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.139848 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-host-cni-bin\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.140085 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.139873 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98g9q\" (UniqueName: \"kubernetes.io/projected/b5a657a4-5088-4c35-96dc-78d8b4a9289f-kube-api-access-98g9q\") pod \"node-resolver-mzzbg\" (UID: \"b5a657a4-5088-4c35-96dc-78d8b4a9289f\") " pod="openshift-dns/node-resolver-mzzbg" Jul 08 22:14:31.140085 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.139919 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/15388137-d252-4a28-996c-3f0ca3236e35-etc-sysctl-d\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.140085 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.139942 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-host-run-ovn-kubernetes\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.140085 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.139950 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 22:14:31.140393 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.139970 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4928f3d7-5f78-4844-9103-cb23d99e9883-ovn-node-metrics-cert\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.140393 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.140258 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/15388137-d252-4a28-996c-3f0ca3236e35-lib-modules\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.140393 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.140285 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54kqs\" (UniqueName: \"kubernetes.io/projected/4928f3d7-5f78-4844-9103-cb23d99e9883-kube-api-access-54kqs\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.140393 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.140312 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v58f\" (UniqueName: \"kubernetes.io/projected/e722d997-2e47-40ac-bb5c-48e25dde7cfd-kube-api-access-4v58f\") pod \"multus-additional-cni-plugins-fbm5c\" (UID: \"e722d997-2e47-40ac-bb5c-48e25dde7cfd\") " pod="openshift-multus/multus-additional-cni-plugins-fbm5c" Jul 08 22:14:31.140393 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.140349 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/15388137-d252-4a28-996c-3f0ca3236e35-etc-sysctl-conf\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.140393 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.140375 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4z27z\" (UniqueName: \"kubernetes.io/projected/c8c71659-1efa-481c-9524-2ebf0b684b97-kube-api-access-4z27z\") pod \"iptables-alerter-45nvq\" (UID: \"c8c71659-1efa-481c-9524-2ebf0b684b97\") " pod="openshift-network-operator/iptables-alerter-45nvq" Jul 08 22:14:31.140689 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.140409 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e722d997-2e47-40ac-bb5c-48e25dde7cfd-cnibin\") pod \"multus-additional-cni-plugins-fbm5c\" (UID: \"e722d997-2e47-40ac-bb5c-48e25dde7cfd\") " pod="openshift-multus/multus-additional-cni-plugins-fbm5c" Jul 08 22:14:31.140689 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.140431 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e722d997-2e47-40ac-bb5c-48e25dde7cfd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fbm5c\" (UID: \"e722d997-2e47-40ac-bb5c-48e25dde7cfd\") " pod="openshift-multus/multus-additional-cni-plugins-fbm5c" Jul 08 22:14:31.140689 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.140454 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/15388137-d252-4a28-996c-3f0ca3236e35-tmp\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.140689 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.140477 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzdh6\" (UniqueName: \"kubernetes.io/projected/15388137-d252-4a28-996c-3f0ca3236e35-kube-api-access-nzdh6\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.140689 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.140500 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-host-cni-netd\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.140689 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.140536 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/15388137-d252-4a28-996c-3f0ca3236e35-etc-systemd\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.140689 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.140563 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b0e80baf-e49e-4b91-ab27-dbc8dc3c2dc4-host\") pod \"node-ca-4rvb8\" (UID: \"b0e80baf-e49e-4b91-ab27-dbc8dc3c2dc4\") " pod="openshift-image-registry/node-ca-4rvb8" Jul 08 22:14:31.140689 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.140639 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b0e80baf-e49e-4b91-ab27-dbc8dc3c2dc4-serviceca\") pod \"node-ca-4rvb8\" (UID: \"b0e80baf-e49e-4b91-ab27-dbc8dc3c2dc4\") " pod="openshift-image-registry/node-ca-4rvb8" Jul 08 22:14:31.141049 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.140677 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c8c71659-1efa-481c-9524-2ebf0b684b97-iptables-alerter-script\") pod \"iptables-alerter-45nvq\" (UID: \"c8c71659-1efa-481c-9524-2ebf0b684b97\") " pod="openshift-network-operator/iptables-alerter-45nvq" Jul 08 22:14:31.141049 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.140737 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c8c71659-1efa-481c-9524-2ebf0b684b97-host-slash\") pod \"iptables-alerter-45nvq\" (UID: \"c8c71659-1efa-481c-9524-2ebf0b684b97\") " pod="openshift-network-operator/iptables-alerter-45nvq" Jul 08 22:14:31.141049 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.141042 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b5a657a4-5088-4c35-96dc-78d8b4a9289f-hosts-file\") pod \"node-resolver-mzzbg\" (UID: \"b5a657a4-5088-4c35-96dc-78d8b4a9289f\") " pod="openshift-dns/node-resolver-mzzbg" Jul 08 22:14:31.141188 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.141072 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/15388137-d252-4a28-996c-3f0ca3236e35-etc-tuned\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.141188 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.141112 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-log-socket\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.141188 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.141135 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5a657a4-5088-4c35-96dc-78d8b4a9289f-tmp-dir\") pod \"node-resolver-mzzbg\" (UID: \"b5a657a4-5088-4c35-96dc-78d8b4a9289f\") " pod="openshift-dns/node-resolver-mzzbg" Jul 08 22:14:31.141188 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.141164 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-host-kubelet\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.141188 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.141185 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-host-slash\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.141418 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.141212 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4928f3d7-5f78-4844-9103-cb23d99e9883-ovnkube-config\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.141418 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.141239 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-run-systemd\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.141418 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.141262 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/15388137-d252-4a28-996c-3f0ca3236e35-etc-sysconfig\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.141418 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.141285 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e722d997-2e47-40ac-bb5c-48e25dde7cfd-os-release\") pod \"multus-additional-cni-plugins-fbm5c\" (UID: \"e722d997-2e47-40ac-bb5c-48e25dde7cfd\") " pod="openshift-multus/multus-additional-cni-plugins-fbm5c" Jul 08 22:14:31.141418 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.141322 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e722d997-2e47-40ac-bb5c-48e25dde7cfd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fbm5c\" (UID: \"e722d997-2e47-40ac-bb5c-48e25dde7cfd\") " pod="openshift-multus/multus-additional-cni-plugins-fbm5c" Jul 08 22:14:31.164084 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.164044 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-x99f7" Jul 08 22:14:31.178077 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.178048 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-x99f7" Jul 08 22:14:31.228444 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.228412 2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 08 22:14:31.241601 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.241567 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e722d997-2e47-40ac-bb5c-48e25dde7cfd-cnibin\") pod \"multus-additional-cni-plugins-fbm5c\" (UID: \"e722d997-2e47-40ac-bb5c-48e25dde7cfd\") " pod="openshift-multus/multus-additional-cni-plugins-fbm5c" Jul 08 22:14:31.241793 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.241609 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e722d997-2e47-40ac-bb5c-48e25dde7cfd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fbm5c\" (UID: \"e722d997-2e47-40ac-bb5c-48e25dde7cfd\") " pod="openshift-multus/multus-additional-cni-plugins-fbm5c" Jul 08 22:14:31.241793 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.241633 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/15388137-d252-4a28-996c-3f0ca3236e35-tmp\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.241793 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.241654 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzdh6\" (UniqueName: \"kubernetes.io/projected/15388137-d252-4a28-996c-3f0ca3236e35-kube-api-access-nzdh6\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.241793 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.241674 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-host-cni-netd\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.241793 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.241700 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zljbc\" (UniqueName: \"kubernetes.io/projected/7c5c6b58-e07d-4d72-af33-ebe169402d4e-kube-api-access-zljbc\") pod \"network-metrics-daemon-jhpj6\" (UID: \"7c5c6b58-e07d-4d72-af33-ebe169402d4e\") " pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:14:31.241793 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.241723 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-etc-kubernetes\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.241793 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.241758 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/15388137-d252-4a28-996c-3f0ca3236e35-etc-systemd\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.241793 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.241782 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b0e80baf-e49e-4b91-ab27-dbc8dc3c2dc4-host\") pod \"node-ca-4rvb8\" (UID: \"b0e80baf-e49e-4b91-ab27-dbc8dc3c2dc4\") " pod="openshift-image-registry/node-ca-4rvb8" Jul 08 22:14:31.242242 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.241804 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b0e80baf-e49e-4b91-ab27-dbc8dc3c2dc4-serviceca\") pod \"node-ca-4rvb8\" (UID: \"b0e80baf-e49e-4b91-ab27-dbc8dc3c2dc4\") " pod="openshift-image-registry/node-ca-4rvb8" Jul 08 22:14:31.242242 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.241842 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c8c71659-1efa-481c-9524-2ebf0b684b97-iptables-alerter-script\") pod \"iptables-alerter-45nvq\" (UID: \"c8c71659-1efa-481c-9524-2ebf0b684b97\") " pod="openshift-network-operator/iptables-alerter-45nvq" Jul 08 22:14:31.242242 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.241868 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c8c71659-1efa-481c-9524-2ebf0b684b97-host-slash\") pod \"iptables-alerter-45nvq\" (UID: \"c8c71659-1efa-481c-9524-2ebf0b684b97\") " pod="openshift-network-operator/iptables-alerter-45nvq" Jul 08 22:14:31.242242 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.241910 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-system-cni-dir\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.242242 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.241985 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/15388137-d252-4a28-996c-3f0ca3236e35-etc-systemd\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.242242 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.242031 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b5a657a4-5088-4c35-96dc-78d8b4a9289f-hosts-file\") pod \"node-resolver-mzzbg\" (UID: \"b5a657a4-5088-4c35-96dc-78d8b4a9289f\") " pod="openshift-dns/node-resolver-mzzbg" Jul 08 22:14:31.242242 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.242056 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/15388137-d252-4a28-996c-3f0ca3236e35-etc-tuned\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.242242 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.242061 2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 08 22:14:31.242242 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.242080 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-log-socket\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.242242 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.242105 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-cnibin\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.242242 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.242130 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ec6d07d3-a0df-413e-abcb-f87ffc795c7d-registration-dir\") pod \"aws-ebs-csi-driver-node-vfvvt\" (UID: \"ec6d07d3-a0df-413e-abcb-f87ffc795c7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfvvt" Jul 08 22:14:31.242242 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.242183 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5a657a4-5088-4c35-96dc-78d8b4a9289f-tmp-dir\") pod \"node-resolver-mzzbg\" (UID: \"b5a657a4-5088-4c35-96dc-78d8b4a9289f\") " pod="openshift-dns/node-resolver-mzzbg" Jul 08 22:14:31.242242 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.242226 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-host-cni-netd\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.242848 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.242289 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-host-kubelet\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.242848 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.242295 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e722d997-2e47-40ac-bb5c-48e25dde7cfd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fbm5c\" (UID: \"e722d997-2e47-40ac-bb5c-48e25dde7cfd\") " pod="openshift-multus/multus-additional-cni-plugins-fbm5c" Jul 08 22:14:31.242848 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.242320 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-host-slash\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.242848 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.242349 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4928f3d7-5f78-4844-9103-cb23d99e9883-ovnkube-config\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.242848 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.242379 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-multus-socket-dir-parent\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.242848 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.242409 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-host-var-lib-cni-multus\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.242848 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.242434 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ec6d07d3-a0df-413e-abcb-f87ffc795c7d-sys-fs\") pod \"aws-ebs-csi-driver-node-vfvvt\" (UID: \"ec6d07d3-a0df-413e-abcb-f87ffc795c7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfvvt" Jul 08 22:14:31.242848 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.242458 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-run-systemd\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.242848 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.242480 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/15388137-d252-4a28-996c-3f0ca3236e35-etc-sysconfig\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.242848 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.242487 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5a657a4-5088-4c35-96dc-78d8b4a9289f-tmp-dir\") pod \"node-resolver-mzzbg\" (UID: \"b5a657a4-5088-4c35-96dc-78d8b4a9289f\") " pod="openshift-dns/node-resolver-mzzbg" Jul 08 22:14:31.242848 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.242503 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e722d997-2e47-40ac-bb5c-48e25dde7cfd-os-release\") pod \"multus-additional-cni-plugins-fbm5c\" (UID: \"e722d997-2e47-40ac-bb5c-48e25dde7cfd\") " pod="openshift-multus/multus-additional-cni-plugins-fbm5c" Jul 08 22:14:31.242848 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.242527 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e722d997-2e47-40ac-bb5c-48e25dde7cfd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fbm5c\" (UID: \"e722d997-2e47-40ac-bb5c-48e25dde7cfd\") " pod="openshift-multus/multus-additional-cni-plugins-fbm5c" Jul 08 22:14:31.242848 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.242553 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-host-run-multus-certs\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.242848 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.242561 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-log-socket\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.242848 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.242577 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-run-ovn\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.242848 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.241669 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e722d997-2e47-40ac-bb5c-48e25dde7cfd-cnibin\") pod \"multus-additional-cni-plugins-fbm5c\" (UID: \"e722d997-2e47-40ac-bb5c-48e25dde7cfd\") " pod="openshift-multus/multus-additional-cni-plugins-fbm5c" Jul 08 22:14:31.242848 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.242603 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4928f3d7-5f78-4844-9103-cb23d99e9883-ovnkube-script-lib\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.243602 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.242630 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e722d997-2e47-40ac-bb5c-48e25dde7cfd-cni-binary-copy\") pod \"multus-additional-cni-plugins-fbm5c\" (UID: \"e722d997-2e47-40ac-bb5c-48e25dde7cfd\") " pod="openshift-multus/multus-additional-cni-plugins-fbm5c" Jul 08 22:14:31.243602 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.242640 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c8c71659-1efa-481c-9524-2ebf0b684b97-iptables-alerter-script\") pod \"iptables-alerter-45nvq\" (UID: \"c8c71659-1efa-481c-9524-2ebf0b684b97\") " pod="openshift-network-operator/iptables-alerter-45nvq" Jul 08 22:14:31.243602 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.242646 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b0e80baf-e49e-4b91-ab27-dbc8dc3c2dc4-host\") pod \"node-ca-4rvb8\" (UID: \"b0e80baf-e49e-4b91-ab27-dbc8dc3c2dc4\") " pod="openshift-image-registry/node-ca-4rvb8" Jul 08 22:14:31.243602 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.242687 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-host-slash\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.243602 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.242721 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-run-systemd\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.243602 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.242768 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/15388137-d252-4a28-996c-3f0ca3236e35-etc-sysconfig\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.243602 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.242824 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-run-ovn\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.243602 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.242832 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e722d997-2e47-40ac-bb5c-48e25dde7cfd-os-release\") pod \"multus-additional-cni-plugins-fbm5c\" (UID: \"e722d997-2e47-40ac-bb5c-48e25dde7cfd\") " pod="openshift-multus/multus-additional-cni-plugins-fbm5c" Jul 08 22:14:31.243602 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.242866 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c8c71659-1efa-481c-9524-2ebf0b684b97-host-slash\") pod \"iptables-alerter-45nvq\" (UID: \"c8c71659-1efa-481c-9524-2ebf0b684b97\") " pod="openshift-network-operator/iptables-alerter-45nvq" Jul 08 22:14:31.243602 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.242915 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b5a657a4-5088-4c35-96dc-78d8b4a9289f-hosts-file\") pod \"node-resolver-mzzbg\" (UID: \"b5a657a4-5088-4c35-96dc-78d8b4a9289f\") " pod="openshift-dns/node-resolver-mzzbg" Jul 08 22:14:31.243602 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.242959 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/15388137-d252-4a28-996c-3f0ca3236e35-sys\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.243602 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.242997 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-etc-openvswitch\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.243602 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243004 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/15388137-d252-4a28-996c-3f0ca3236e35-sys\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.243602 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243035 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e722d997-2e47-40ac-bb5c-48e25dde7cfd-system-cni-dir\") pod \"multus-additional-cni-plugins-fbm5c\" (UID: \"e722d997-2e47-40ac-bb5c-48e25dde7cfd\") " pod="openshift-multus/multus-additional-cni-plugins-fbm5c" Jul 08 22:14:31.243602 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243044 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-etc-openvswitch\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.243602 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243078 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b0e80baf-e49e-4b91-ab27-dbc8dc3c2dc4-serviceca\") pod \"node-ca-4rvb8\" (UID: \"b0e80baf-e49e-4b91-ab27-dbc8dc3c2dc4\") " pod="openshift-image-registry/node-ca-4rvb8" Jul 08 22:14:31.243602 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243088 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-host-run-k8s-cni-cncf-io\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.243602 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243118 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-host-var-lib-cni-bin\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.244404 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243140 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e722d997-2e47-40ac-bb5c-48e25dde7cfd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fbm5c\" (UID: \"e722d997-2e47-40ac-bb5c-48e25dde7cfd\") " pod="openshift-multus/multus-additional-cni-plugins-fbm5c" Jul 08 22:14:31.244404 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243148 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e722d997-2e47-40ac-bb5c-48e25dde7cfd-system-cni-dir\") pod \"multus-additional-cni-plugins-fbm5c\" (UID: \"e722d997-2e47-40ac-bb5c-48e25dde7cfd\") " pod="openshift-multus/multus-additional-cni-plugins-fbm5c" Jul 08 22:14:31.244404 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243176 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-host-kubelet\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.244404 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243188 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e1de91b1-aa3f-4288-9b87-718980dcea7e-konnectivity-ca\") pod \"konnectivity-agent-p4pz6\" (UID: \"e1de91b1-aa3f-4288-9b87-718980dcea7e\") " pod="kube-system/konnectivity-agent-p4pz6" Jul 08 22:14:31.244404 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243214 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ec6d07d3-a0df-413e-abcb-f87ffc795c7d-socket-dir\") pod \"aws-ebs-csi-driver-node-vfvvt\" (UID: \"ec6d07d3-a0df-413e-abcb-f87ffc795c7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfvvt" Jul 08 22:14:31.244404 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243257 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4928f3d7-5f78-4844-9103-cb23d99e9883-ovnkube-config\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.244404 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243279 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ec6d07d3-a0df-413e-abcb-f87ffc795c7d-etc-selinux\") pod \"aws-ebs-csi-driver-node-vfvvt\" (UID: \"ec6d07d3-a0df-413e-abcb-f87ffc795c7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfvvt" Jul 08 22:14:31.244404 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243305 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmgj9\" (UniqueName: \"kubernetes.io/projected/b0e80baf-e49e-4b91-ab27-dbc8dc3c2dc4-kube-api-access-kmgj9\") pod \"node-ca-4rvb8\" (UID: \"b0e80baf-e49e-4b91-ab27-dbc8dc3c2dc4\") " pod="openshift-image-registry/node-ca-4rvb8" Jul 08 22:14:31.244404 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243330 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e722d997-2e47-40ac-bb5c-48e25dde7cfd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fbm5c\" (UID: \"e722d997-2e47-40ac-bb5c-48e25dde7cfd\") " pod="openshift-multus/multus-additional-cni-plugins-fbm5c" Jul 08 22:14:31.244404 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243344 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4928f3d7-5f78-4844-9103-cb23d99e9883-ovnkube-script-lib\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.244404 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243355 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnbkg\" (UniqueName: \"kubernetes.io/projected/6a151365-c622-4361-862f-ed202f3045f7-kube-api-access-mnbkg\") pod \"network-check-target-98x2n\" (UID: \"6a151365-c622-4361-862f-ed202f3045f7\") " pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:14:31.244404 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243380 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-systemd-units\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.244404 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243406 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/15388137-d252-4a28-996c-3f0ca3236e35-etc-modprobe-d\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.244404 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243417 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-systemd-units\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.244404 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243429 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/15388137-d252-4a28-996c-3f0ca3236e35-etc-kubernetes\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.244404 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243455 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.244404 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243481 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4a354dc2-56c0-4280-96d1-b560d70be2b2-kubelet-config\") pod \"global-pull-secret-syncer-fffds\" (UID: \"4a354dc2-56c0-4280-96d1-b560d70be2b2\") " pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:14:31.245227 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243505 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/15388137-d252-4a28-996c-3f0ca3236e35-run\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.245227 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243511 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e722d997-2e47-40ac-bb5c-48e25dde7cfd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fbm5c\" (UID: \"e722d997-2e47-40ac-bb5c-48e25dde7cfd\") " pod="openshift-multus/multus-additional-cni-plugins-fbm5c" Jul 08 22:14:31.245227 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243510 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e722d997-2e47-40ac-bb5c-48e25dde7cfd-cni-binary-copy\") pod \"multus-additional-cni-plugins-fbm5c\" (UID: \"e722d997-2e47-40ac-bb5c-48e25dde7cfd\") " pod="openshift-multus/multus-additional-cni-plugins-fbm5c" Jul 08 22:14:31.245227 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243529 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/15388137-d252-4a28-996c-3f0ca3236e35-etc-modprobe-d\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.245227 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243531 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/15388137-d252-4a28-996c-3f0ca3236e35-etc-kubernetes\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.245227 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243532 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/15388137-d252-4a28-996c-3f0ca3236e35-var-lib-kubelet\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.245227 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243553 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.245227 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243571 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/15388137-d252-4a28-996c-3f0ca3236e35-host\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.245227 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243582 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/15388137-d252-4a28-996c-3f0ca3236e35-run\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.245227 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243595 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-node-log\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.245227 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243618 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4928f3d7-5f78-4844-9103-cb23d99e9883-env-overrides\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.245227 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243626 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/15388137-d252-4a28-996c-3f0ca3236e35-host\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.245227 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243645 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-multus-cni-dir\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.245227 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243653 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4a354dc2-56c0-4280-96d1-b560d70be2b2-kubelet-config\") pod \"global-pull-secret-syncer-fffds\" (UID: \"4a354dc2-56c0-4280-96d1-b560d70be2b2\") " pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:14:31.245227 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243659 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-node-log\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.245227 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243628 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/15388137-d252-4a28-996c-3f0ca3236e35-var-lib-kubelet\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.245227 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243671 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-host-var-lib-kubelet\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.245227 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243702 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4a354dc2-56c0-4280-96d1-b560d70be2b2-dbus\") pod \"global-pull-secret-syncer-fffds\" (UID: \"4a354dc2-56c0-4280-96d1-b560d70be2b2\") " pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:14:31.246090 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243726 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a354dc2-56c0-4280-96d1-b560d70be2b2-original-pull-secret\") pod \"global-pull-secret-syncer-fffds\" (UID: \"4a354dc2-56c0-4280-96d1-b560d70be2b2\") " pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:14:31.246090 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243780 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-var-lib-openvswitch\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.246090 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243806 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-run-openvswitch\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.246090 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:31.243817 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 22:14:31.246090 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243831 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-os-release\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.246090 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243853 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-multus-conf-dir\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.246090 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243864 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-var-lib-openvswitch\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.246090 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243876 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v97jb\" (UniqueName: \"kubernetes.io/projected/3dbef84e-5232-42ba-9727-c632df781129-kube-api-access-v97jb\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.246090 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243830 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4a354dc2-56c0-4280-96d1-b560d70be2b2-dbus\") pod \"global-pull-secret-syncer-fffds\" (UID: \"4a354dc2-56c0-4280-96d1-b560d70be2b2\") " pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:14:31.246090 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:31.243911 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a354dc2-56c0-4280-96d1-b560d70be2b2-original-pull-secret podName:4a354dc2-56c0-4280-96d1-b560d70be2b2 nodeName:}" failed. No retries permitted until 2026-07-08 22:14:31.743875092 +0000 UTC m=+3.011895114 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a354dc2-56c0-4280-96d1-b560d70be2b2-original-pull-secret") pod "global-pull-secret-syncer-fffds" (UID: "4a354dc2-56c0-4280-96d1-b560d70be2b2") : object "kube-system"/"original-pull-secret" not registered Jul 08 22:14:31.246090 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243951 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-run-openvswitch\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.246090 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.243983 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ec6d07d3-a0df-413e-abcb-f87ffc795c7d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vfvvt\" (UID: \"ec6d07d3-a0df-413e-abcb-f87ffc795c7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfvvt" Jul 08 22:14:31.246090 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.244017 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-host-run-netns\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.246090 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.244046 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-host-cni-bin\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.246090 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.244073 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c5c6b58-e07d-4d72-af33-ebe169402d4e-metrics-certs\") pod \"network-metrics-daemon-jhpj6\" (UID: \"7c5c6b58-e07d-4d72-af33-ebe169402d4e\") " pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:14:31.246090 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.244077 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4928f3d7-5f78-4844-9103-cb23d99e9883-env-overrides\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.246090 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.244077 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-host-run-netns\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.246983 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.244093 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-hostroot\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.246983 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.244113 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-host-cni-bin\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.246983 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.244117 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98g9q\" (UniqueName: \"kubernetes.io/projected/b5a657a4-5088-4c35-96dc-78d8b4a9289f-kube-api-access-98g9q\") pod \"node-resolver-mzzbg\" (UID: \"b5a657a4-5088-4c35-96dc-78d8b4a9289f\") " pod="openshift-dns/node-resolver-mzzbg" Jul 08 22:14:31.246983 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.244158 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/15388137-d252-4a28-996c-3f0ca3236e35-etc-sysctl-d\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.246983 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.244186 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-host-run-ovn-kubernetes\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.246983 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.244212 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4928f3d7-5f78-4844-9103-cb23d99e9883-ovn-node-metrics-cert\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.246983 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.244240 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3dbef84e-5232-42ba-9727-c632df781129-cni-binary-copy\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.246983 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.244251 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4928f3d7-5f78-4844-9103-cb23d99e9883-host-run-ovn-kubernetes\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.246983 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.244265 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-host-run-netns\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.246983 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.244267 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/15388137-d252-4a28-996c-3f0ca3236e35-etc-sysctl-d\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.246983 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.244292 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e1de91b1-aa3f-4288-9b87-718980dcea7e-agent-certs\") pod \"konnectivity-agent-p4pz6\" (UID: \"e1de91b1-aa3f-4288-9b87-718980dcea7e\") " pod="kube-system/konnectivity-agent-p4pz6" Jul 08 22:14:31.246983 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.244319 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ec6d07d3-a0df-413e-abcb-f87ffc795c7d-device-dir\") pod \"aws-ebs-csi-driver-node-vfvvt\" (UID: \"ec6d07d3-a0df-413e-abcb-f87ffc795c7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfvvt" Jul 08 22:14:31.246983 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.244344 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2df6\" (UniqueName: \"kubernetes.io/projected/ec6d07d3-a0df-413e-abcb-f87ffc795c7d-kube-api-access-b2df6\") pod \"aws-ebs-csi-driver-node-vfvvt\" (UID: \"ec6d07d3-a0df-413e-abcb-f87ffc795c7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfvvt" Jul 08 22:14:31.246983 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.244370 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/15388137-d252-4a28-996c-3f0ca3236e35-lib-modules\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.246983 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.244396 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54kqs\" (UniqueName: \"kubernetes.io/projected/4928f3d7-5f78-4844-9103-cb23d99e9883-kube-api-access-54kqs\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.246983 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.244421 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v58f\" (UniqueName: \"kubernetes.io/projected/e722d997-2e47-40ac-bb5c-48e25dde7cfd-kube-api-access-4v58f\") pod \"multus-additional-cni-plugins-fbm5c\" (UID: \"e722d997-2e47-40ac-bb5c-48e25dde7cfd\") " pod="openshift-multus/multus-additional-cni-plugins-fbm5c" Jul 08 22:14:31.246983 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.244520 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3dbef84e-5232-42ba-9727-c632df781129-multus-daemon-config\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.247797 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.244526 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/15388137-d252-4a28-996c-3f0ca3236e35-lib-modules\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.247797 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.244569 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/15388137-d252-4a28-996c-3f0ca3236e35-etc-sysctl-conf\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.247797 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.244615 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z27z\" (UniqueName: \"kubernetes.io/projected/c8c71659-1efa-481c-9524-2ebf0b684b97-kube-api-access-4z27z\") pod \"iptables-alerter-45nvq\" (UID: \"c8c71659-1efa-481c-9524-2ebf0b684b97\") " pod="openshift-network-operator/iptables-alerter-45nvq" Jul 08 22:14:31.247797 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.244770 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/15388137-d252-4a28-996c-3f0ca3236e35-etc-sysctl-conf\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.247797 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.245976 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/15388137-d252-4a28-996c-3f0ca3236e35-etc-tuned\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.247797 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.246045 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/15388137-d252-4a28-996c-3f0ca3236e35-tmp\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.248130 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.248109 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4928f3d7-5f78-4844-9103-cb23d99e9883-ovn-node-metrics-cert\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.250802 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.250776 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzdh6\" (UniqueName: \"kubernetes.io/projected/15388137-d252-4a28-996c-3f0ca3236e35-kube-api-access-nzdh6\") pod \"tuned-bj85s\" (UID: \"15388137-d252-4a28-996c-3f0ca3236e35\") " pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.251721 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.251699 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmgj9\" (UniqueName: \"kubernetes.io/projected/b0e80baf-e49e-4b91-ab27-dbc8dc3c2dc4-kube-api-access-kmgj9\") pod \"node-ca-4rvb8\" (UID: \"b0e80baf-e49e-4b91-ab27-dbc8dc3c2dc4\") " pod="openshift-image-registry/node-ca-4rvb8" Jul 08 22:14:31.254107 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.254084 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-54kqs\" (UniqueName: \"kubernetes.io/projected/4928f3d7-5f78-4844-9103-cb23d99e9883-kube-api-access-54kqs\") pod \"ovnkube-node-bltvn\" (UID: \"4928f3d7-5f78-4844-9103-cb23d99e9883\") " pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.254355 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.254334 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98g9q\" (UniqueName: \"kubernetes.io/projected/b5a657a4-5088-4c35-96dc-78d8b4a9289f-kube-api-access-98g9q\") pod \"node-resolver-mzzbg\" (UID: \"b5a657a4-5088-4c35-96dc-78d8b4a9289f\") " pod="openshift-dns/node-resolver-mzzbg" Jul 08 22:14:31.255145 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.255124 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v58f\" (UniqueName: \"kubernetes.io/projected/e722d997-2e47-40ac-bb5c-48e25dde7cfd-kube-api-access-4v58f\") pod \"multus-additional-cni-plugins-fbm5c\" (UID: \"e722d997-2e47-40ac-bb5c-48e25dde7cfd\") " pod="openshift-multus/multus-additional-cni-plugins-fbm5c" Jul 08 22:14:31.256583 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.256556 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4z27z\" (UniqueName: \"kubernetes.io/projected/c8c71659-1efa-481c-9524-2ebf0b684b97-kube-api-access-4z27z\") pod \"iptables-alerter-45nvq\" (UID: \"c8c71659-1efa-481c-9524-2ebf0b684b97\") " pod="openshift-network-operator/iptables-alerter-45nvq" Jul 08 22:14:31.317066 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.317004 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-162.ec2.internal" event={"ID":"b96734b96bcdda2fdee4b1945ad78b59","Type":"ContainerStarted","Data":"897fab4aee666fdb6acb7191a20afd09a685c79d84a58aa25ffebfc6fe1a36fb"} Jul 08 22:14:31.318166 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.318136 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-162.ec2.internal" event={"ID":"55e574bf7ca2ba660429fe3c53ea934c","Type":"ContainerStarted","Data":"feb29f034f55c18062e4bd1224f3cddb5047afb363ebd3c827436f8de84537be"} Jul 08 22:14:31.345389 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.345355 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3dbef84e-5232-42ba-9727-c632df781129-cni-binary-copy\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.345389 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.345391 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-host-run-netns\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.345697 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.345410 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e1de91b1-aa3f-4288-9b87-718980dcea7e-agent-certs\") pod \"konnectivity-agent-p4pz6\" (UID: \"e1de91b1-aa3f-4288-9b87-718980dcea7e\") " pod="kube-system/konnectivity-agent-p4pz6" Jul 08 22:14:31.345697 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.345426 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ec6d07d3-a0df-413e-abcb-f87ffc795c7d-device-dir\") pod \"aws-ebs-csi-driver-node-vfvvt\" (UID: \"ec6d07d3-a0df-413e-abcb-f87ffc795c7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfvvt" Jul 08 22:14:31.345697 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.345486 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ec6d07d3-a0df-413e-abcb-f87ffc795c7d-device-dir\") pod \"aws-ebs-csi-driver-node-vfvvt\" (UID: \"ec6d07d3-a0df-413e-abcb-f87ffc795c7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfvvt" Jul 08 22:14:31.345697 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.345487 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-host-run-netns\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.345697 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.345537 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2df6\" (UniqueName: \"kubernetes.io/projected/ec6d07d3-a0df-413e-abcb-f87ffc795c7d-kube-api-access-b2df6\") pod \"aws-ebs-csi-driver-node-vfvvt\" (UID: \"ec6d07d3-a0df-413e-abcb-f87ffc795c7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfvvt" Jul 08 22:14:31.345697 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.345566 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3dbef84e-5232-42ba-9727-c632df781129-multus-daemon-config\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.345697 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.345590 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zljbc\" (UniqueName: \"kubernetes.io/projected/7c5c6b58-e07d-4d72-af33-ebe169402d4e-kube-api-access-zljbc\") pod \"network-metrics-daemon-jhpj6\" (UID: \"7c5c6b58-e07d-4d72-af33-ebe169402d4e\") " pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:14:31.345697 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.345615 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-etc-kubernetes\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.345697 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.345644 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-system-cni-dir\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.345697 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.345688 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-cnibin\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.346194 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.345716 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ec6d07d3-a0df-413e-abcb-f87ffc795c7d-registration-dir\") pod \"aws-ebs-csi-driver-node-vfvvt\" (UID: \"ec6d07d3-a0df-413e-abcb-f87ffc795c7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfvvt" Jul 08 22:14:31.346194 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.345745 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-multus-socket-dir-parent\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.346194 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.345769 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-host-var-lib-cni-multus\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.346194 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.345796 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ec6d07d3-a0df-413e-abcb-f87ffc795c7d-sys-fs\") pod \"aws-ebs-csi-driver-node-vfvvt\" (UID: \"ec6d07d3-a0df-413e-abcb-f87ffc795c7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfvvt" Jul 08 22:14:31.346194 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.345859 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ec6d07d3-a0df-413e-abcb-f87ffc795c7d-registration-dir\") pod \"aws-ebs-csi-driver-node-vfvvt\" (UID: \"ec6d07d3-a0df-413e-abcb-f87ffc795c7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfvvt" Jul 08 22:14:31.346194 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.345928 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-etc-kubernetes\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.346194 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.345863 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ec6d07d3-a0df-413e-abcb-f87ffc795c7d-sys-fs\") pod \"aws-ebs-csi-driver-node-vfvvt\" (UID: \"ec6d07d3-a0df-413e-abcb-f87ffc795c7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfvvt" Jul 08 22:14:31.346194 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.345929 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-host-var-lib-cni-multus\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.346194 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.345959 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-multus-socket-dir-parent\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.346194 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.345857 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-system-cni-dir\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.346194 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.345977 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-host-run-multus-certs\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.346194 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.346013 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-host-run-k8s-cni-cncf-io\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.346194 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.346035 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-cnibin\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.346194 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.346038 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3dbef84e-5232-42ba-9727-c632df781129-cni-binary-copy\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.346194 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.346057 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-host-var-lib-cni-bin\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.346194 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.346083 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-host-run-multus-certs\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.346194 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.346093 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-host-var-lib-cni-bin\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.346194 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.346096 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e1de91b1-aa3f-4288-9b87-718980dcea7e-konnectivity-ca\") pod \"konnectivity-agent-p4pz6\" (UID: \"e1de91b1-aa3f-4288-9b87-718980dcea7e\") " pod="kube-system/konnectivity-agent-p4pz6" Jul 08 22:14:31.346993 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.346098 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3dbef84e-5232-42ba-9727-c632df781129-multus-daemon-config\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.346993 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.346124 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ec6d07d3-a0df-413e-abcb-f87ffc795c7d-socket-dir\") pod \"aws-ebs-csi-driver-node-vfvvt\" (UID: \"ec6d07d3-a0df-413e-abcb-f87ffc795c7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfvvt" Jul 08 22:14:31.346993 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.346142 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-host-run-k8s-cni-cncf-io\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.346993 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.346176 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ec6d07d3-a0df-413e-abcb-f87ffc795c7d-etc-selinux\") pod \"aws-ebs-csi-driver-node-vfvvt\" (UID: \"ec6d07d3-a0df-413e-abcb-f87ffc795c7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfvvt" Jul 08 22:14:31.346993 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.346210 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ec6d07d3-a0df-413e-abcb-f87ffc795c7d-socket-dir\") pod \"aws-ebs-csi-driver-node-vfvvt\" (UID: \"ec6d07d3-a0df-413e-abcb-f87ffc795c7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfvvt" Jul 08 22:14:31.346993 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.346339 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ec6d07d3-a0df-413e-abcb-f87ffc795c7d-etc-selinux\") pod \"aws-ebs-csi-driver-node-vfvvt\" (UID: \"ec6d07d3-a0df-413e-abcb-f87ffc795c7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfvvt" Jul 08 22:14:31.346993 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.346584 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e1de91b1-aa3f-4288-9b87-718980dcea7e-konnectivity-ca\") pod \"konnectivity-agent-p4pz6\" (UID: \"e1de91b1-aa3f-4288-9b87-718980dcea7e\") " pod="kube-system/konnectivity-agent-p4pz6" Jul 08 22:14:31.346993 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.346619 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnbkg\" (UniqueName: \"kubernetes.io/projected/6a151365-c622-4361-862f-ed202f3045f7-kube-api-access-mnbkg\") pod \"network-check-target-98x2n\" (UID: \"6a151365-c622-4361-862f-ed202f3045f7\") " pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:14:31.346993 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.346652 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-multus-cni-dir\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.346993 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.346671 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-host-var-lib-kubelet\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.346993 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.346709 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-os-release\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.346993 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.346731 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-multus-conf-dir\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.346993 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.346744 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-multus-cni-dir\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.346993 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.346753 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v97jb\" (UniqueName: \"kubernetes.io/projected/3dbef84e-5232-42ba-9727-c632df781129-kube-api-access-v97jb\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.346993 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.346776 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ec6d07d3-a0df-413e-abcb-f87ffc795c7d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vfvvt\" (UID: \"ec6d07d3-a0df-413e-abcb-f87ffc795c7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfvvt" Jul 08 22:14:31.346993 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.346794 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-host-var-lib-kubelet\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.346993 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.346803 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c5c6b58-e07d-4d72-af33-ebe169402d4e-metrics-certs\") pod \"network-metrics-daemon-jhpj6\" (UID: \"7c5c6b58-e07d-4d72-af33-ebe169402d4e\") " pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:14:31.346993 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.346834 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-os-release\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.347746 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.346850 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-multus-conf-dir\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.347746 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.346867 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-hostroot\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.347746 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.346836 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3dbef84e-5232-42ba-9727-c632df781129-hostroot\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.347746 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:31.346964 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 22:14:31.347746 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.346930 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ec6d07d3-a0df-413e-abcb-f87ffc795c7d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vfvvt\" (UID: \"ec6d07d3-a0df-413e-abcb-f87ffc795c7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfvvt" Jul 08 22:14:31.347746 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:31.347327 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c5c6b58-e07d-4d72-af33-ebe169402d4e-metrics-certs podName:7c5c6b58-e07d-4d72-af33-ebe169402d4e nodeName:}" failed. No retries permitted until 2026-07-08 22:14:31.847305005 +0000 UTC m=+3.115325029 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c5c6b58-e07d-4d72-af33-ebe169402d4e-metrics-certs") pod "network-metrics-daemon-jhpj6" (UID: "7c5c6b58-e07d-4d72-af33-ebe169402d4e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 22:14:31.350429 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.350407 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e1de91b1-aa3f-4288-9b87-718980dcea7e-agent-certs\") pod \"konnectivity-agent-p4pz6\" (UID: \"e1de91b1-aa3f-4288-9b87-718980dcea7e\") " pod="kube-system/konnectivity-agent-p4pz6" Jul 08 22:14:31.353577 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.353553 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2df6\" (UniqueName: \"kubernetes.io/projected/ec6d07d3-a0df-413e-abcb-f87ffc795c7d-kube-api-access-b2df6\") pod \"aws-ebs-csi-driver-node-vfvvt\" (UID: \"ec6d07d3-a0df-413e-abcb-f87ffc795c7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfvvt" Jul 08 22:14:31.353697 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.353557 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zljbc\" (UniqueName: \"kubernetes.io/projected/7c5c6b58-e07d-4d72-af33-ebe169402d4e-kube-api-access-zljbc\") pod \"network-metrics-daemon-jhpj6\" (UID: \"7c5c6b58-e07d-4d72-af33-ebe169402d4e\") " pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:14:31.354731 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:31.354713 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 22:14:31.354804 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:31.354740 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 22:14:31.354804 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:31.354753 2563 projected.go:196] Error preparing data for projected volume kube-api-access-mnbkg for pod openshift-network-diagnostics/network-check-target-98x2n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 22:14:31.354868 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:31.354816 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a151365-c622-4361-862f-ed202f3045f7-kube-api-access-mnbkg podName:6a151365-c622-4361-862f-ed202f3045f7 nodeName:}" failed. No retries permitted until 2026-07-08 22:14:31.854801357 +0000 UTC m=+3.122821364 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-mnbkg" (UniqueName: "kubernetes.io/projected/6a151365-c622-4361-862f-ed202f3045f7-kube-api-access-mnbkg") pod "network-check-target-98x2n" (UID: "6a151365-c622-4361-862f-ed202f3045f7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 22:14:31.356802 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.356780 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v97jb\" (UniqueName: \"kubernetes.io/projected/3dbef84e-5232-42ba-9727-c632df781129-kube-api-access-v97jb\") pod \"multus-8fgq2\" (UID: \"3dbef84e-5232-42ba-9727-c632df781129\") " pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.432334 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.432227 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mzzbg" Jul 08 22:14:31.441631 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.441601 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bj85s" Jul 08 22:14:31.452961 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.452929 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-45nvq" Jul 08 22:14:31.458136 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.458084 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:31.465900 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.465866 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fbm5c" Jul 08 22:14:31.476313 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.476284 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4rvb8" Jul 08 22:14:31.486198 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.486167 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8fgq2" Jul 08 22:14:31.493007 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.492979 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-p4pz6" Jul 08 22:14:31.499921 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.499896 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfvvt" Jul 08 22:14:31.750445 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.750409 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a354dc2-56c0-4280-96d1-b560d70be2b2-original-pull-secret\") pod \"global-pull-secret-syncer-fffds\" (UID: \"4a354dc2-56c0-4280-96d1-b560d70be2b2\") " pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:14:31.750659 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:31.750582 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 22:14:31.750659 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:31.750657 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a354dc2-56c0-4280-96d1-b560d70be2b2-original-pull-secret podName:4a354dc2-56c0-4280-96d1-b560d70be2b2 nodeName:}" failed. No retries permitted until 2026-07-08 22:14:32.750640726 +0000 UTC m=+4.018660732 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a354dc2-56c0-4280-96d1-b560d70be2b2-original-pull-secret") pod "global-pull-secret-syncer-fffds" (UID: "4a354dc2-56c0-4280-96d1-b560d70be2b2") : object "kube-system"/"original-pull-secret" not registered Jul 08 22:14:31.851564 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.851524 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c5c6b58-e07d-4d72-af33-ebe169402d4e-metrics-certs\") pod \"network-metrics-daemon-jhpj6\" (UID: \"7c5c6b58-e07d-4d72-af33-ebe169402d4e\") " pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:14:31.851732 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:31.851648 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 22:14:31.851732 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:31.851705 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c5c6b58-e07d-4d72-af33-ebe169402d4e-metrics-certs podName:7c5c6b58-e07d-4d72-af33-ebe169402d4e nodeName:}" failed. No retries permitted until 2026-07-08 22:14:32.851686614 +0000 UTC m=+4.119706633 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c5c6b58-e07d-4d72-af33-ebe169402d4e-metrics-certs") pod "network-metrics-daemon-jhpj6" (UID: "7c5c6b58-e07d-4d72-af33-ebe169402d4e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 22:14:31.952802 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:31.952767 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnbkg\" (UniqueName: \"kubernetes.io/projected/6a151365-c622-4361-862f-ed202f3045f7-kube-api-access-mnbkg\") pod \"network-check-target-98x2n\" (UID: \"6a151365-c622-4361-862f-ed202f3045f7\") " pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:14:31.952994 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:31.952946 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 22:14:31.952994 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:31.952969 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 22:14:31.952994 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:31.952984 2563 projected.go:196] Error preparing data for projected volume kube-api-access-mnbkg for pod openshift-network-diagnostics/network-check-target-98x2n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 22:14:31.953171 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:31.953047 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a151365-c622-4361-862f-ed202f3045f7-kube-api-access-mnbkg podName:6a151365-c622-4361-862f-ed202f3045f7 nodeName:}" failed. No retries permitted until 2026-07-08 22:14:32.953030461 +0000 UTC m=+4.221050484 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-mnbkg" (UniqueName: "kubernetes.io/projected/6a151365-c622-4361-862f-ed202f3045f7-kube-api-access-mnbkg") pod "network-check-target-98x2n" (UID: "6a151365-c622-4361-862f-ed202f3045f7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 22:14:32.181729 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:32.181668 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 22:09:31 +0000 UTC" deadline="2028-01-22 02:41:54.166789853 +0000 UTC" Jul 08 22:14:32.181729 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:32.181725 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13492h27m21.985069605s" Jul 08 22:14:32.313079 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:32.313041 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:14:32.313252 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:32.313056 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:14:32.313252 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:32.313187 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98x2n" podUID="6a151365-c622-4361-862f-ed202f3045f7" Jul 08 22:14:32.313252 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:32.313239 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhpj6" podUID="7c5c6b58-e07d-4d72-af33-ebe169402d4e" Jul 08 22:14:32.399663 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:32.399635 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5a657a4_5088_4c35_96dc_78d8b4a9289f.slice/crio-e8c576751e23ace7744d9cc685cc91da3d8c056a562e0abc131989aba0257a35 WatchSource:0}: Error finding container e8c576751e23ace7744d9cc685cc91da3d8c056a562e0abc131989aba0257a35: Status 404 returned error can't find the container with id e8c576751e23ace7744d9cc685cc91da3d8c056a562e0abc131989aba0257a35 Jul 08 22:14:32.400709 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:32.400681 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0e80baf_e49e_4b91_ab27_dbc8dc3c2dc4.slice/crio-d4d2149da38f56fbf93ef86528d928c42492174c8f70bfcb5bd089caa58833be WatchSource:0}: Error finding container d4d2149da38f56fbf93ef86528d928c42492174c8f70bfcb5bd089caa58833be: Status 404 returned error can't find the container with id d4d2149da38f56fbf93ef86528d928c42492174c8f70bfcb5bd089caa58833be Jul 08 22:14:32.401738 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:32.401565 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1de91b1_aa3f_4288_9b87_718980dcea7e.slice/crio-430bce69a2d2410c7df05cb43f6d2e8ede9b0c61b7634ece548b3a9f3ae02e8e WatchSource:0}: Error finding container 430bce69a2d2410c7df05cb43f6d2e8ede9b0c61b7634ece548b3a9f3ae02e8e: Status 404 returned error can't find the container with id 430bce69a2d2410c7df05cb43f6d2e8ede9b0c61b7634ece548b3a9f3ae02e8e Jul 08 22:14:32.404997 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:32.404960 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8c71659_1efa_481c_9524_2ebf0b684b97.slice/crio-0e3c265735891a5f7b8b2b68acd906bda71b0dc733418dadaa320286ec716ba5 WatchSource:0}: Error finding container 0e3c265735891a5f7b8b2b68acd906bda71b0dc733418dadaa320286ec716ba5: Status 404 returned error can't find the container with id 0e3c265735891a5f7b8b2b68acd906bda71b0dc733418dadaa320286ec716ba5 Jul 08 22:14:32.405622 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:32.405584 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode722d997_2e47_40ac_bb5c_48e25dde7cfd.slice/crio-0dcaa7076b53119a9063947123fda1aa8b31b1e57608d3eef7c0b7eaeb7e7db3 WatchSource:0}: Error finding container 0dcaa7076b53119a9063947123fda1aa8b31b1e57608d3eef7c0b7eaeb7e7db3: Status 404 returned error can't find the container with id 0dcaa7076b53119a9063947123fda1aa8b31b1e57608d3eef7c0b7eaeb7e7db3 Jul 08 22:14:32.406957 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:32.406936 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4928f3d7_5f78_4844_9103_cb23d99e9883.slice/crio-f3f5275d613e7dc9bd6a59f58189e533af736199a416259d2f0442cc9d7b3cd9 WatchSource:0}: Error finding container f3f5275d613e7dc9bd6a59f58189e533af736199a416259d2f0442cc9d7b3cd9: Status 404 returned error can't find the container with id f3f5275d613e7dc9bd6a59f58189e533af736199a416259d2f0442cc9d7b3cd9 Jul 08 22:14:32.408267 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:32.408101 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15388137_d252_4a28_996c_3f0ca3236e35.slice/crio-f6722b6417ed0dff83383acb83e4474f2531825e242f42d67fae55e4aa85e085 WatchSource:0}: Error finding container f6722b6417ed0dff83383acb83e4474f2531825e242f42d67fae55e4aa85e085: Status 404 returned error can't find the container with id f6722b6417ed0dff83383acb83e4474f2531825e242f42d67fae55e4aa85e085 Jul 08 22:14:32.409604 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:32.409504 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec6d07d3_a0df_413e_abcb_f87ffc795c7d.slice/crio-bde6263ae8ce2376d7cedc5d5687fa99baf73e521d915b53c79f1de0f20eda12 WatchSource:0}: Error finding container bde6263ae8ce2376d7cedc5d5687fa99baf73e521d915b53c79f1de0f20eda12: Status 404 returned error can't find the container with id bde6263ae8ce2376d7cedc5d5687fa99baf73e521d915b53c79f1de0f20eda12 Jul 08 22:14:32.411175 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:14:32.410820 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dbef84e_5232_42ba_9727_c632df781129.slice/crio-9582ee47bedbedace24804d16f4c10ea832c0e35c11fa8726ec3e16d7fb1baa8 WatchSource:0}: Error finding container 9582ee47bedbedace24804d16f4c10ea832c0e35c11fa8726ec3e16d7fb1baa8: Status 404 returned error can't find the container with id 9582ee47bedbedace24804d16f4c10ea832c0e35c11fa8726ec3e16d7fb1baa8 Jul 08 22:14:32.759225 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:32.759018 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a354dc2-56c0-4280-96d1-b560d70be2b2-original-pull-secret\") pod \"global-pull-secret-syncer-fffds\" (UID: \"4a354dc2-56c0-4280-96d1-b560d70be2b2\") " pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:14:32.759370 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:32.759118 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 22:14:32.759414 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:32.759393 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a354dc2-56c0-4280-96d1-b560d70be2b2-original-pull-secret podName:4a354dc2-56c0-4280-96d1-b560d70be2b2 nodeName:}" failed. No retries permitted until 2026-07-08 22:14:34.759372443 +0000 UTC m=+6.027392466 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a354dc2-56c0-4280-96d1-b560d70be2b2-original-pull-secret") pod "global-pull-secret-syncer-fffds" (UID: "4a354dc2-56c0-4280-96d1-b560d70be2b2") : object "kube-system"/"original-pull-secret" not registered Jul 08 22:14:32.860225 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:32.860150 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c5c6b58-e07d-4d72-af33-ebe169402d4e-metrics-certs\") pod \"network-metrics-daemon-jhpj6\" (UID: \"7c5c6b58-e07d-4d72-af33-ebe169402d4e\") " pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:14:32.860389 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:32.860293 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 22:14:32.860389 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:32.860368 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c5c6b58-e07d-4d72-af33-ebe169402d4e-metrics-certs podName:7c5c6b58-e07d-4d72-af33-ebe169402d4e nodeName:}" failed. No retries permitted until 2026-07-08 22:14:34.860348158 +0000 UTC m=+6.128368189 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c5c6b58-e07d-4d72-af33-ebe169402d4e-metrics-certs") pod "network-metrics-daemon-jhpj6" (UID: "7c5c6b58-e07d-4d72-af33-ebe169402d4e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 22:14:32.961068 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:32.960619 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnbkg\" (UniqueName: \"kubernetes.io/projected/6a151365-c622-4361-862f-ed202f3045f7-kube-api-access-mnbkg\") pod \"network-check-target-98x2n\" (UID: \"6a151365-c622-4361-862f-ed202f3045f7\") " pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:14:32.961068 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:32.960817 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 22:14:32.961068 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:32.960837 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 22:14:32.961068 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:32.960850 2563 projected.go:196] Error preparing data for projected volume kube-api-access-mnbkg for pod openshift-network-diagnostics/network-check-target-98x2n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 22:14:32.961068 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:32.960924 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a151365-c622-4361-862f-ed202f3045f7-kube-api-access-mnbkg podName:6a151365-c622-4361-862f-ed202f3045f7 nodeName:}" failed. No retries permitted until 2026-07-08 22:14:34.960905401 +0000 UTC m=+6.228925412 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-mnbkg" (UniqueName: "kubernetes.io/projected/6a151365-c622-4361-862f-ed202f3045f7-kube-api-access-mnbkg") pod "network-check-target-98x2n" (UID: "6a151365-c622-4361-862f-ed202f3045f7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 22:14:33.182284 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:33.182236 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 22:09:31 +0000 UTC" deadline="2028-04-24 00:03:07.334436932 +0000 UTC" Jul 08 22:14:33.182284 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:33.182281 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15721h48m34.152161091s" Jul 08 22:14:33.315115 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:33.314603 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:14:33.315115 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:33.314737 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fffds" podUID="4a354dc2-56c0-4280-96d1-b560d70be2b2" Jul 08 22:14:33.338075 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:33.337984 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfvvt" event={"ID":"ec6d07d3-a0df-413e-abcb-f87ffc795c7d","Type":"ContainerStarted","Data":"bde6263ae8ce2376d7cedc5d5687fa99baf73e521d915b53c79f1de0f20eda12"} Jul 08 22:14:33.344879 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:33.344839 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" event={"ID":"4928f3d7-5f78-4844-9103-cb23d99e9883","Type":"ContainerStarted","Data":"f3f5275d613e7dc9bd6a59f58189e533af736199a416259d2f0442cc9d7b3cd9"} Jul 08 22:14:33.354119 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:33.354079 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4rvb8" event={"ID":"b0e80baf-e49e-4b91-ab27-dbc8dc3c2dc4","Type":"ContainerStarted","Data":"d4d2149da38f56fbf93ef86528d928c42492174c8f70bfcb5bd089caa58833be"} Jul 08 22:14:33.369460 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:33.369415 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mzzbg" event={"ID":"b5a657a4-5088-4c35-96dc-78d8b4a9289f","Type":"ContainerStarted","Data":"e8c576751e23ace7744d9cc685cc91da3d8c056a562e0abc131989aba0257a35"} Jul 08 22:14:33.377741 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:33.377652 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8fgq2" event={"ID":"3dbef84e-5232-42ba-9727-c632df781129","Type":"ContainerStarted","Data":"9582ee47bedbedace24804d16f4c10ea832c0e35c11fa8726ec3e16d7fb1baa8"} Jul 08 22:14:33.383164 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:33.383062 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bj85s" event={"ID":"15388137-d252-4a28-996c-3f0ca3236e35","Type":"ContainerStarted","Data":"f6722b6417ed0dff83383acb83e4474f2531825e242f42d67fae55e4aa85e085"} Jul 08 22:14:33.395372 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:33.395283 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fbm5c" event={"ID":"e722d997-2e47-40ac-bb5c-48e25dde7cfd","Type":"ContainerStarted","Data":"0dcaa7076b53119a9063947123fda1aa8b31b1e57608d3eef7c0b7eaeb7e7db3"} Jul 08 22:14:33.402634 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:33.402534 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-45nvq" event={"ID":"c8c71659-1efa-481c-9524-2ebf0b684b97","Type":"ContainerStarted","Data":"0e3c265735891a5f7b8b2b68acd906bda71b0dc733418dadaa320286ec716ba5"} Jul 08 22:14:33.405141 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:33.405107 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-p4pz6" event={"ID":"e1de91b1-aa3f-4288-9b87-718980dcea7e","Type":"ContainerStarted","Data":"430bce69a2d2410c7df05cb43f6d2e8ede9b0c61b7634ece548b3a9f3ae02e8e"} Jul 08 22:14:33.411006 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:33.410865 2563 generic.go:356] "Generic (PLEG): container finished" podID="b96734b96bcdda2fdee4b1945ad78b59" containerID="5c3e83f93ab871ed6bc36e8bbca3791ba28bd24b3c533cd95dffc2a18dd2c2ca" exitCode=0 Jul 08 22:14:33.411166 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:33.411078 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-162.ec2.internal" event={"ID":"b96734b96bcdda2fdee4b1945ad78b59","Type":"ContainerDied","Data":"5c3e83f93ab871ed6bc36e8bbca3791ba28bd24b3c533cd95dffc2a18dd2c2ca"} Jul 08 22:14:33.438073 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:33.438019 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-162.ec2.internal" event={"ID":"55e574bf7ca2ba660429fe3c53ea934c","Type":"ContainerStarted","Data":"1905c10635132cf6584ff646875cf6449ece0b611b421d0e58685f58b8222014"} Jul 08 22:14:34.313046 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:34.313009 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:14:34.313498 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:34.313144 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98x2n" podUID="6a151365-c622-4361-862f-ed202f3045f7" Jul 08 22:14:34.313576 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:34.313552 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:14:34.313707 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:34.313669 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhpj6" podUID="7c5c6b58-e07d-4d72-af33-ebe169402d4e" Jul 08 22:14:34.472928 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:34.472820 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-162.ec2.internal" event={"ID":"b96734b96bcdda2fdee4b1945ad78b59","Type":"ContainerStarted","Data":"b56b0e4005312bf4070e01b234ff2a9126a3b98453708616df240ec506cd1f8f"} Jul 08 22:14:34.490021 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:34.489954 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-162.ec2.internal" podStartSLOduration=4.489927195 podStartE2EDuration="4.489927195s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 22:14:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:14:33.454900271 +0000 UTC m=+4.722920292" watchObservedRunningTime="2026-07-08 22:14:34.489927195 +0000 UTC m=+5.757947225" Jul 08 22:14:34.778325 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:34.778189 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a354dc2-56c0-4280-96d1-b560d70be2b2-original-pull-secret\") pod \"global-pull-secret-syncer-fffds\" (UID: \"4a354dc2-56c0-4280-96d1-b560d70be2b2\") " pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:14:34.778504 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:34.778379 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 22:14:34.778504 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:34.778453 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a354dc2-56c0-4280-96d1-b560d70be2b2-original-pull-secret podName:4a354dc2-56c0-4280-96d1-b560d70be2b2 nodeName:}" failed. No retries permitted until 2026-07-08 22:14:38.778432162 +0000 UTC m=+10.046452173 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a354dc2-56c0-4280-96d1-b560d70be2b2-original-pull-secret") pod "global-pull-secret-syncer-fffds" (UID: "4a354dc2-56c0-4280-96d1-b560d70be2b2") : object "kube-system"/"original-pull-secret" not registered Jul 08 22:14:34.879964 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:34.879931 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c5c6b58-e07d-4d72-af33-ebe169402d4e-metrics-certs\") pod \"network-metrics-daemon-jhpj6\" (UID: \"7c5c6b58-e07d-4d72-af33-ebe169402d4e\") " pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:14:34.880133 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:34.880069 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 22:14:34.880133 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:34.880127 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c5c6b58-e07d-4d72-af33-ebe169402d4e-metrics-certs podName:7c5c6b58-e07d-4d72-af33-ebe169402d4e nodeName:}" failed. No retries permitted until 2026-07-08 22:14:38.880110029 +0000 UTC m=+10.148130037 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c5c6b58-e07d-4d72-af33-ebe169402d4e-metrics-certs") pod "network-metrics-daemon-jhpj6" (UID: "7c5c6b58-e07d-4d72-af33-ebe169402d4e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 22:14:34.980950 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:34.980898 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnbkg\" (UniqueName: \"kubernetes.io/projected/6a151365-c622-4361-862f-ed202f3045f7-kube-api-access-mnbkg\") pod \"network-check-target-98x2n\" (UID: \"6a151365-c622-4361-862f-ed202f3045f7\") " pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:14:34.981170 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:34.981151 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 22:14:34.981232 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:34.981180 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 22:14:34.981232 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:34.981194 2563 projected.go:196] Error preparing data for projected volume kube-api-access-mnbkg for pod openshift-network-diagnostics/network-check-target-98x2n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 22:14:34.981335 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:34.981254 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a151365-c622-4361-862f-ed202f3045f7-kube-api-access-mnbkg podName:6a151365-c622-4361-862f-ed202f3045f7 nodeName:}" failed. No retries permitted until 2026-07-08 22:14:38.981236275 +0000 UTC m=+10.249256286 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-mnbkg" (UniqueName: "kubernetes.io/projected/6a151365-c622-4361-862f-ed202f3045f7-kube-api-access-mnbkg") pod "network-check-target-98x2n" (UID: "6a151365-c622-4361-862f-ed202f3045f7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 22:14:35.313103 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:35.313070 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:14:35.313552 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:35.313211 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fffds" podUID="4a354dc2-56c0-4280-96d1-b560d70be2b2" Jul 08 22:14:36.312625 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:36.312564 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:14:36.312809 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:36.312714 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98x2n" podUID="6a151365-c622-4361-862f-ed202f3045f7" Jul 08 22:14:36.313144 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:36.313123 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:14:36.313532 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:36.313232 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhpj6" podUID="7c5c6b58-e07d-4d72-af33-ebe169402d4e" Jul 08 22:14:37.313248 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:37.313215 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:14:37.313731 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:37.313465 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fffds" podUID="4a354dc2-56c0-4280-96d1-b560d70be2b2" Jul 08 22:14:38.313244 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:38.313200 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:14:38.313244 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:38.313233 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:14:38.313766 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:38.313360 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98x2n" podUID="6a151365-c622-4361-862f-ed202f3045f7" Jul 08 22:14:38.313766 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:38.313495 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhpj6" podUID="7c5c6b58-e07d-4d72-af33-ebe169402d4e" Jul 08 22:14:38.811878 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:38.811839 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a354dc2-56c0-4280-96d1-b560d70be2b2-original-pull-secret\") pod \"global-pull-secret-syncer-fffds\" (UID: \"4a354dc2-56c0-4280-96d1-b560d70be2b2\") " pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:14:38.812096 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:38.812043 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 22:14:38.812163 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:38.812155 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a354dc2-56c0-4280-96d1-b560d70be2b2-original-pull-secret podName:4a354dc2-56c0-4280-96d1-b560d70be2b2 nodeName:}" failed. No retries permitted until 2026-07-08 22:14:46.812133272 +0000 UTC m=+18.080153289 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a354dc2-56c0-4280-96d1-b560d70be2b2-original-pull-secret") pod "global-pull-secret-syncer-fffds" (UID: "4a354dc2-56c0-4280-96d1-b560d70be2b2") : object "kube-system"/"original-pull-secret" not registered Jul 08 22:14:38.912320 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:38.912284 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c5c6b58-e07d-4d72-af33-ebe169402d4e-metrics-certs\") pod \"network-metrics-daemon-jhpj6\" (UID: \"7c5c6b58-e07d-4d72-af33-ebe169402d4e\") " pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:14:38.912531 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:38.912430 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 22:14:38.912531 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:38.912511 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c5c6b58-e07d-4d72-af33-ebe169402d4e-metrics-certs podName:7c5c6b58-e07d-4d72-af33-ebe169402d4e nodeName:}" failed. No retries permitted until 2026-07-08 22:14:46.912490716 +0000 UTC m=+18.180510728 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c5c6b58-e07d-4d72-af33-ebe169402d4e-metrics-certs") pod "network-metrics-daemon-jhpj6" (UID: "7c5c6b58-e07d-4d72-af33-ebe169402d4e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 22:14:39.012857 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:39.012815 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnbkg\" (UniqueName: \"kubernetes.io/projected/6a151365-c622-4361-862f-ed202f3045f7-kube-api-access-mnbkg\") pod \"network-check-target-98x2n\" (UID: \"6a151365-c622-4361-862f-ed202f3045f7\") " pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:14:39.013074 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:39.013055 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 22:14:39.013136 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:39.013083 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 22:14:39.013136 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:39.013099 2563 projected.go:196] Error preparing data for projected volume kube-api-access-mnbkg for pod openshift-network-diagnostics/network-check-target-98x2n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 22:14:39.013243 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:39.013161 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a151365-c622-4361-862f-ed202f3045f7-kube-api-access-mnbkg podName:6a151365-c622-4361-862f-ed202f3045f7 nodeName:}" failed. No retries permitted until 2026-07-08 22:14:47.013142296 +0000 UTC m=+18.281162325 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-mnbkg" (UniqueName: "kubernetes.io/projected/6a151365-c622-4361-862f-ed202f3045f7-kube-api-access-mnbkg") pod "network-check-target-98x2n" (UID: "6a151365-c622-4361-862f-ed202f3045f7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 22:14:39.313609 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:39.313569 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:14:39.314094 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:39.313694 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fffds" podUID="4a354dc2-56c0-4280-96d1-b560d70be2b2" Jul 08 22:14:40.313103 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:40.313070 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:14:40.313310 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:40.313081 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:14:40.313310 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:40.313206 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98x2n" podUID="6a151365-c622-4361-862f-ed202f3045f7" Jul 08 22:14:40.313310 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:40.313289 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhpj6" podUID="7c5c6b58-e07d-4d72-af33-ebe169402d4e" Jul 08 22:14:41.312348 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:41.312318 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:14:41.312828 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:41.312448 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fffds" podUID="4a354dc2-56c0-4280-96d1-b560d70be2b2" Jul 08 22:14:42.313226 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:42.313195 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:14:42.313700 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:42.313195 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:14:42.313700 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:42.313329 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98x2n" podUID="6a151365-c622-4361-862f-ed202f3045f7" Jul 08 22:14:42.313700 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:42.313429 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhpj6" podUID="7c5c6b58-e07d-4d72-af33-ebe169402d4e" Jul 08 22:14:43.312978 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:43.312935 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:14:43.313120 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:43.313086 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fffds" podUID="4a354dc2-56c0-4280-96d1-b560d70be2b2" Jul 08 22:14:43.499144 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:43.498914 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfvvt" event={"ID":"ec6d07d3-a0df-413e-abcb-f87ffc795c7d","Type":"ContainerStarted","Data":"1e321aa5e9b380c945ccc6b4aec2fd51bfc9406f00f7ba796116fbc09b338794"} Jul 08 22:14:43.500759 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:43.500728 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4rvb8" event={"ID":"b0e80baf-e49e-4b91-ab27-dbc8dc3c2dc4","Type":"ContainerStarted","Data":"e8aacae401eddb14c94576fc2519a85f8795c021bb561670e75db49602045117"} Jul 08 22:14:43.502349 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:43.502320 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mzzbg" event={"ID":"b5a657a4-5088-4c35-96dc-78d8b4a9289f","Type":"ContainerStarted","Data":"0ecbf3b137faf19c591ce96d7b76234dbdf9ee3e78c933e19e41ef555aca4029"} Jul 08 22:14:43.503962 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:43.503931 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bj85s" event={"ID":"15388137-d252-4a28-996c-3f0ca3236e35","Type":"ContainerStarted","Data":"c53e4bb2a22cec2c33b521450a05f7dc59d202d223336161c36203fe621c5498"} Jul 08 22:14:43.505672 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:43.505644 2563 generic.go:356] "Generic (PLEG): container finished" podID="e722d997-2e47-40ac-bb5c-48e25dde7cfd" containerID="bf26d9815ff4002f6c8e2c1fc8746192b32413f10449aa4774cad821bd6d1ab9" exitCode=0 Jul 08 22:14:43.505780 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:43.505726 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fbm5c" event={"ID":"e722d997-2e47-40ac-bb5c-48e25dde7cfd","Type":"ContainerDied","Data":"bf26d9815ff4002f6c8e2c1fc8746192b32413f10449aa4774cad821bd6d1ab9"} Jul 08 22:14:43.507372 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:43.507315 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-p4pz6" event={"ID":"e1de91b1-aa3f-4288-9b87-718980dcea7e","Type":"ContainerStarted","Data":"ad9be1adf6dc2a1b2b05cbc5d5815e01619f1c6a9882149756806511cf5abd76"} Jul 08 22:14:43.517195 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:43.517144 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4rvb8" podStartSLOduration=4.259220687 podStartE2EDuration="14.517129452s" totalImagesPullingTime="10.257908765s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 22:14:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:14:43.516419864 +0000 UTC m=+14.784439893" watchObservedRunningTime="2026-07-08 22:14:43.517129452 +0000 UTC m=+14.785149481" Jul 08 22:14:43.517367 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:43.517228 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-162.ec2.internal" podStartSLOduration=13.517222193 podStartE2EDuration="13.517222193s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 22:14:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:14:34.490653827 +0000 UTC m=+5.758673856" watchObservedRunningTime="2026-07-08 22:14:43.517222193 +0000 UTC m=+14.785242221" Jul 08 22:14:43.530809 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:43.530760 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mzzbg" podStartSLOduration=4.265005377 podStartE2EDuration="14.530745903s" totalImagesPullingTime="10.265740526s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 22:14:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:14:43.530049187 +0000 UTC m=+14.798069214" watchObservedRunningTime="2026-07-08 22:14:43.530745903 +0000 UTC m=+14.798765930" Jul 08 22:14:43.570300 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:43.570229 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-bj85s" podStartSLOduration=4.066689494 podStartE2EDuration="14.570207254s" totalImagesPullingTime="10.50351776s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 22:14:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:14:43.570062629 +0000 UTC m=+14.838082659" watchObservedRunningTime="2026-07-08 22:14:43.570207254 +0000 UTC m=+14.838227283" Jul 08 22:14:43.586150 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:43.586092 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-p4pz6" podStartSLOduration=4.344239345 podStartE2EDuration="14.586070943s" totalImagesPullingTime="10.241831598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 22:14:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:14:43.585525036 +0000 UTC m=+14.853545067" watchObservedRunningTime="2026-07-08 22:14:43.586070943 +0000 UTC m=+14.854090974" Jul 08 22:14:44.313107 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:44.313067 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:14:44.313107 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:44.313106 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:14:44.313315 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:44.313194 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98x2n" podUID="6a151365-c622-4361-862f-ed202f3045f7" Jul 08 22:14:44.313362 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:44.313320 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhpj6" podUID="7c5c6b58-e07d-4d72-af33-ebe169402d4e" Jul 08 22:14:44.511009 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:44.510938 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-45nvq" event={"ID":"c8c71659-1efa-481c-9524-2ebf0b684b97","Type":"ContainerStarted","Data":"e26fabfba166a5799a854b419abb32f188a8d9bf90b6f9f08e680e71e65ce023"} Jul 08 22:14:44.526943 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:44.526865 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-45nvq" podStartSLOduration=5.271542947 podStartE2EDuration="15.526842581s" totalImagesPullingTime="10.255299634s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 22:14:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:14:44.526710883 +0000 UTC m=+15.794730912" watchObservedRunningTime="2026-07-08 22:14:44.526842581 +0000 UTC m=+15.794862614" Jul 08 22:14:45.312396 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:45.312361 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:14:45.312584 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:45.312492 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fffds" podUID="4a354dc2-56c0-4280-96d1-b560d70be2b2" Jul 08 22:14:46.312719 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:46.312683 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:14:46.312719 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:46.312712 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:14:46.313373 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:46.312813 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98x2n" podUID="6a151365-c622-4361-862f-ed202f3045f7" Jul 08 22:14:46.313373 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:46.312951 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhpj6" podUID="7c5c6b58-e07d-4d72-af33-ebe169402d4e" Jul 08 22:14:46.828614 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:46.828584 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mzzbg_b5a657a4-5088-4c35-96dc-78d8b4a9289f/dns-node-resolver/0.log" Jul 08 22:14:46.872981 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:46.872942 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a354dc2-56c0-4280-96d1-b560d70be2b2-original-pull-secret\") pod \"global-pull-secret-syncer-fffds\" (UID: \"4a354dc2-56c0-4280-96d1-b560d70be2b2\") " pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:14:46.873135 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:46.873074 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 22:14:46.873193 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:46.873142 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a354dc2-56c0-4280-96d1-b560d70be2b2-original-pull-secret podName:4a354dc2-56c0-4280-96d1-b560d70be2b2 nodeName:}" failed. No retries permitted until 2026-07-08 22:15:02.87312752 +0000 UTC m=+34.141147525 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a354dc2-56c0-4280-96d1-b560d70be2b2-original-pull-secret") pod "global-pull-secret-syncer-fffds" (UID: "4a354dc2-56c0-4280-96d1-b560d70be2b2") : object "kube-system"/"original-pull-secret" not registered Jul 08 22:14:46.973868 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:46.973827 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c5c6b58-e07d-4d72-af33-ebe169402d4e-metrics-certs\") pod \"network-metrics-daemon-jhpj6\" (UID: \"7c5c6b58-e07d-4d72-af33-ebe169402d4e\") " pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:14:46.974044 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:46.974018 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 22:14:46.974115 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:46.974097 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c5c6b58-e07d-4d72-af33-ebe169402d4e-metrics-certs podName:7c5c6b58-e07d-4d72-af33-ebe169402d4e nodeName:}" failed. No retries permitted until 2026-07-08 22:15:02.974074803 +0000 UTC m=+34.242094815 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c5c6b58-e07d-4d72-af33-ebe169402d4e-metrics-certs") pod "network-metrics-daemon-jhpj6" (UID: "7c5c6b58-e07d-4d72-af33-ebe169402d4e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 22:14:47.074532 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:47.074494 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnbkg\" (UniqueName: \"kubernetes.io/projected/6a151365-c622-4361-862f-ed202f3045f7-kube-api-access-mnbkg\") pod \"network-check-target-98x2n\" (UID: \"6a151365-c622-4361-862f-ed202f3045f7\") " pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:14:47.074718 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:47.074695 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 22:14:47.074789 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:47.074726 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 22:14:47.074789 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:47.074740 2563 projected.go:196] Error preparing data for projected volume kube-api-access-mnbkg for pod openshift-network-diagnostics/network-check-target-98x2n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 22:14:47.074872 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:47.074808 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a151365-c622-4361-862f-ed202f3045f7-kube-api-access-mnbkg podName:6a151365-c622-4361-862f-ed202f3045f7 nodeName:}" failed. No retries permitted until 2026-07-08 22:15:03.074786398 +0000 UTC m=+34.342806407 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-mnbkg" (UniqueName: "kubernetes.io/projected/6a151365-c622-4361-862f-ed202f3045f7-kube-api-access-mnbkg") pod "network-check-target-98x2n" (UID: "6a151365-c622-4361-862f-ed202f3045f7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 22:14:47.094087 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:47.094000 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-p4pz6" Jul 08 22:14:47.094719 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:47.094699 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-p4pz6" Jul 08 22:14:47.312540 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:47.312500 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:14:47.312729 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:47.312639 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fffds" podUID="4a354dc2-56c0-4280-96d1-b560d70be2b2" Jul 08 22:14:47.516692 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:47.516664 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-p4pz6" Jul 08 22:14:47.517411 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:47.517260 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-p4pz6" Jul 08 22:14:48.003919 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:48.003872 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4rvb8_b0e80baf-e49e-4b91-ab27-dbc8dc3c2dc4/node-ca/0.log" Jul 08 22:14:48.312325 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:48.312231 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:14:48.312325 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:48.312265 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:14:48.312542 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:48.312369 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98x2n" podUID="6a151365-c622-4361-862f-ed202f3045f7" Jul 08 22:14:48.312542 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:48.312473 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhpj6" podUID="7c5c6b58-e07d-4d72-af33-ebe169402d4e" Jul 08 22:14:49.318705 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:49.318665 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:14:49.319380 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:49.318946 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fffds" podUID="4a354dc2-56c0-4280-96d1-b560d70be2b2" Jul 08 22:14:50.312553 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:50.312511 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:14:50.312760 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:50.312570 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:14:50.312760 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:50.312660 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98x2n" podUID="6a151365-c622-4361-862f-ed202f3045f7" Jul 08 22:14:50.312868 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:50.312836 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhpj6" podUID="7c5c6b58-e07d-4d72-af33-ebe169402d4e" Jul 08 22:14:51.312646 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:51.312576 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:14:51.313216 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:51.312716 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fffds" podUID="4a354dc2-56c0-4280-96d1-b560d70be2b2" Jul 08 22:14:52.313164 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:52.312976 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:14:52.313619 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:52.313005 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:14:52.313619 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:52.313272 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhpj6" podUID="7c5c6b58-e07d-4d72-af33-ebe169402d4e" Jul 08 22:14:52.313619 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:52.313320 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98x2n" podUID="6a151365-c622-4361-862f-ed202f3045f7" Jul 08 22:14:52.529158 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:52.529119 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8fgq2" event={"ID":"3dbef84e-5232-42ba-9727-c632df781129","Type":"ContainerStarted","Data":"302d933b05238c883bf1525f159f5c2b81668fa7c6e8df3a389aaf340990e377"} Jul 08 22:14:52.530741 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:52.530721 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bltvn_4928f3d7-5f78-4844-9103-cb23d99e9883/ovn-acl-logging/0.log" Jul 08 22:14:52.531068 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:52.531049 2563 generic.go:356] "Generic (PLEG): container finished" podID="4928f3d7-5f78-4844-9103-cb23d99e9883" containerID="630294ff25e9677d71d471f1909d9171908a8fc22c5f919fef835e4b6a5cf128" exitCode=1 Jul 08 22:14:52.531137 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:52.531080 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" event={"ID":"4928f3d7-5f78-4844-9103-cb23d99e9883","Type":"ContainerDied","Data":"630294ff25e9677d71d471f1909d9171908a8fc22c5f919fef835e4b6a5cf128"} Jul 08 22:14:52.531137 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:52.531129 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" event={"ID":"4928f3d7-5f78-4844-9103-cb23d99e9883","Type":"ContainerStarted","Data":"d4a968c4d9f06a4345d438d590bebf5ec97ac220c18190e6ffbd2e4e2abf40ad"} Jul 08 22:14:52.549715 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:52.549651 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-8fgq2" podStartSLOduration=3.828910023 podStartE2EDuration="23.549633087s" totalImagesPullingTime="19.720723064s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 22:14:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:14:52.54921833 +0000 UTC m=+23.817238359" watchObservedRunningTime="2026-07-08 22:14:52.549633087 +0000 UTC m=+23.817653114" Jul 08 22:14:53.312819 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:53.312783 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:14:53.313056 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:53.312930 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fffds" podUID="4a354dc2-56c0-4280-96d1-b560d70be2b2" Jul 08 22:14:54.227205 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:54.227056 2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 08 22:14:54.313095 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:54.313054 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:14:54.313258 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:54.313067 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:14:54.313258 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:54.313187 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhpj6" podUID="7c5c6b58-e07d-4d72-af33-ebe169402d4e" Jul 08 22:14:54.313258 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:54.313232 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98x2n" podUID="6a151365-c622-4361-862f-ed202f3045f7" Jul 08 22:14:54.536952 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:54.536923 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfvvt" event={"ID":"ec6d07d3-a0df-413e-abcb-f87ffc795c7d","Type":"ContainerStarted","Data":"2d3e0c75d714bd46881b16a88ff29c9f4ecd3be914a3fb6bc523d258c874583b"} Jul 08 22:14:54.539687 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:54.539670 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bltvn_4928f3d7-5f78-4844-9103-cb23d99e9883/ovn-acl-logging/0.log" Jul 08 22:14:54.540142 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:54.540115 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" event={"ID":"4928f3d7-5f78-4844-9103-cb23d99e9883","Type":"ContainerStarted","Data":"2adc19473f6bae12ac308071279d68e810e52fda1ad16277f1f5b464e03204d8"} Jul 08 22:14:54.540215 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:54.540186 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" event={"ID":"4928f3d7-5f78-4844-9103-cb23d99e9883","Type":"ContainerStarted","Data":"b5dea20fdf796ab16078d3e7bb024af7dcf602b9c4f87e0227ac62fc479db50b"} Jul 08 22:14:54.540259 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:54.540234 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" event={"ID":"4928f3d7-5f78-4844-9103-cb23d99e9883","Type":"ContainerStarted","Data":"658eb72e7bfb498c62e68c36abd5e072c56331a9b1b1bc866e7be643ebb9b85d"} Jul 08 22:14:54.541954 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:54.541932 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fbm5c" event={"ID":"e722d997-2e47-40ac-bb5c-48e25dde7cfd","Type":"ContainerStarted","Data":"12d398448f814f941dc595c565b9cb442d0fa687c0e8edd10d08152ef7e79ae0"} Jul 08 22:14:55.202904 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:55.202784 2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T22:14:54.227198739Z","UUID":"a812853e-56f5-4957-9748-e89ae1d3ec55","Handler":null,"Name":"","Endpoint":""} Jul 08 22:14:55.204300 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:55.204282 2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 08 22:14:55.204398 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:55.204318 2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 08 22:14:55.312828 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:55.312779 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:14:55.313377 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:55.312915 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fffds" podUID="4a354dc2-56c0-4280-96d1-b560d70be2b2" Jul 08 22:14:55.546969 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:55.546942 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bltvn_4928f3d7-5f78-4844-9103-cb23d99e9883/ovn-acl-logging/0.log" Jul 08 22:14:55.547346 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:55.547317 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" event={"ID":"4928f3d7-5f78-4844-9103-cb23d99e9883","Type":"ContainerStarted","Data":"343fc5ae43210fadc7e15150be5832f73b37aa66aa08140963da43bae9ea0e97"} Jul 08 22:14:55.549253 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:55.549228 2563 generic.go:356] "Generic (PLEG): container finished" podID="e722d997-2e47-40ac-bb5c-48e25dde7cfd" containerID="12d398448f814f941dc595c565b9cb442d0fa687c0e8edd10d08152ef7e79ae0" exitCode=0 Jul 08 22:14:55.549369 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:55.549278 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fbm5c" event={"ID":"e722d997-2e47-40ac-bb5c-48e25dde7cfd","Type":"ContainerDied","Data":"12d398448f814f941dc595c565b9cb442d0fa687c0e8edd10d08152ef7e79ae0"} Jul 08 22:14:56.313150 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:56.313113 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:14:56.313610 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:56.313113 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:14:56.313610 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:56.313232 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhpj6" podUID="7c5c6b58-e07d-4d72-af33-ebe169402d4e" Jul 08 22:14:56.313610 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:56.313293 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98x2n" podUID="6a151365-c622-4361-862f-ed202f3045f7" Jul 08 22:14:56.554062 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:56.554021 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfvvt" event={"ID":"ec6d07d3-a0df-413e-abcb-f87ffc795c7d","Type":"ContainerStarted","Data":"1f4c81136002553534be232c4367f12a545da2a3330990578a1f6711f4c8849c"} Jul 08 22:14:56.572481 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:56.572354 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfvvt" podStartSLOduration=4.439876239 podStartE2EDuration="27.572335233s" totalImagesPullingTime="23.132458994s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 22:14:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:14:56.57108342 +0000 UTC m=+27.839103462" watchObservedRunningTime="2026-07-08 22:14:56.572335233 +0000 UTC m=+27.840355261" Jul 08 22:14:57.312440 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:57.312401 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:14:57.312630 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:57.312563 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fffds" podUID="4a354dc2-56c0-4280-96d1-b560d70be2b2" Jul 08 22:14:57.560520 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:57.560487 2563 generic.go:356] "Generic (PLEG): container finished" podID="e722d997-2e47-40ac-bb5c-48e25dde7cfd" containerID="498ab14c5a056fa591caf6cedf08baa3baf312e5c2c87f2f643ea8abc22bc503" exitCode=0 Jul 08 22:14:57.561073 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:57.560583 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fbm5c" event={"ID":"e722d997-2e47-40ac-bb5c-48e25dde7cfd","Type":"ContainerDied","Data":"498ab14c5a056fa591caf6cedf08baa3baf312e5c2c87f2f643ea8abc22bc503"} Jul 08 22:14:57.566223 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:57.566204 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bltvn_4928f3d7-5f78-4844-9103-cb23d99e9883/ovn-acl-logging/0.log" Jul 08 22:14:57.566609 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:57.566585 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" event={"ID":"4928f3d7-5f78-4844-9103-cb23d99e9883","Type":"ContainerStarted","Data":"12b34ef3d00636944a42ffff3d2e38cd4eedd576717a8672bfef6508dc31e4eb"} Jul 08 22:14:58.312826 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:58.312788 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:14:58.312826 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:58.312822 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:14:58.313053 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:58.312921 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98x2n" podUID="6a151365-c622-4361-862f-ed202f3045f7" Jul 08 22:14:58.313053 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:58.313037 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhpj6" podUID="7c5c6b58-e07d-4d72-af33-ebe169402d4e" Jul 08 22:14:59.313672 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:59.313487 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:14:59.314249 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:14:59.313726 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fffds" podUID="4a354dc2-56c0-4280-96d1-b560d70be2b2" Jul 08 22:14:59.574034 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:59.573945 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bltvn_4928f3d7-5f78-4844-9103-cb23d99e9883/ovn-acl-logging/0.log" Jul 08 22:14:59.574311 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:59.574279 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" event={"ID":"4928f3d7-5f78-4844-9103-cb23d99e9883","Type":"ContainerStarted","Data":"7e08d597ce69ea8c605ca37648c33e3a8b60b396a07c130092591c500c9155cb"} Jul 08 22:14:59.574604 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:59.574578 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:14:59.577803 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:59.575173 2563 scope.go:117] "RemoveContainer" containerID="630294ff25e9677d71d471f1909d9171908a8fc22c5f919fef835e4b6a5cf128" Jul 08 22:14:59.580596 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:59.580573 2563 generic.go:356] "Generic (PLEG): container finished" podID="e722d997-2e47-40ac-bb5c-48e25dde7cfd" containerID="de17c156cb8e4f00253496bd66245868378eb81222ae1f33a6af5e97c3718d87" exitCode=0 Jul 08 22:14:59.580715 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:59.580619 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fbm5c" event={"ID":"e722d997-2e47-40ac-bb5c-48e25dde7cfd","Type":"ContainerDied","Data":"de17c156cb8e4f00253496bd66245868378eb81222ae1f33a6af5e97c3718d87"} Jul 08 22:14:59.593645 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:14:59.593613 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:15:00.312656 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:00.312621 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:15:00.312826 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:00.312622 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:15:00.312826 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:00.312720 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98x2n" podUID="6a151365-c622-4361-862f-ed202f3045f7" Jul 08 22:15:00.312956 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:00.312859 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhpj6" podUID="7c5c6b58-e07d-4d72-af33-ebe169402d4e" Jul 08 22:15:00.587496 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:00.587416 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bltvn_4928f3d7-5f78-4844-9103-cb23d99e9883/ovn-acl-logging/0.log" Jul 08 22:15:00.587932 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:00.587840 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" event={"ID":"4928f3d7-5f78-4844-9103-cb23d99e9883","Type":"ContainerStarted","Data":"8b22c16ea28eb19aa59f3a579a73a6b6afe0c200028b4537ed62bd462705d643"} Jul 08 22:15:00.588261 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:00.588237 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:15:00.588367 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:00.588324 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:15:00.605803 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:00.605779 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:15:00.613359 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:00.613314 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" podStartSLOduration=11.888459365 podStartE2EDuration="31.613295945s" totalImagesPullingTime="19.72483658s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 22:14:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:15:00.611652286 +0000 UTC m=+31.879672306" watchObservedRunningTime="2026-07-08 22:15:00.613295945 +0000 UTC m=+31.881316045" Jul 08 22:15:01.313193 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:01.313156 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:15:01.313377 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:01.313292 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fffds" podUID="4a354dc2-56c0-4280-96d1-b560d70be2b2" Jul 08 22:15:02.313085 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:02.313048 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:15:02.313534 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:02.313062 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:15:02.313534 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:02.313191 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98x2n" podUID="6a151365-c622-4361-862f-ed202f3045f7" Jul 08 22:15:02.313534 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:02.313271 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhpj6" podUID="7c5c6b58-e07d-4d72-af33-ebe169402d4e" Jul 08 22:15:02.897500 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:02.897458 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a354dc2-56c0-4280-96d1-b560d70be2b2-original-pull-secret\") pod \"global-pull-secret-syncer-fffds\" (UID: \"4a354dc2-56c0-4280-96d1-b560d70be2b2\") " pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:15:02.897715 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:02.897607 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 22:15:02.897715 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:02.897680 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a354dc2-56c0-4280-96d1-b560d70be2b2-original-pull-secret podName:4a354dc2-56c0-4280-96d1-b560d70be2b2 nodeName:}" failed. No retries permitted until 2026-07-08 22:15:34.897659489 +0000 UTC m=+66.165679505 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a354dc2-56c0-4280-96d1-b560d70be2b2-original-pull-secret") pod "global-pull-secret-syncer-fffds" (UID: "4a354dc2-56c0-4280-96d1-b560d70be2b2") : object "kube-system"/"original-pull-secret" not registered Jul 08 22:15:02.998693 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:02.998653 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c5c6b58-e07d-4d72-af33-ebe169402d4e-metrics-certs\") pod \"network-metrics-daemon-jhpj6\" (UID: \"7c5c6b58-e07d-4d72-af33-ebe169402d4e\") " pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:15:02.998896 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:02.998789 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 22:15:02.998896 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:02.998857 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c5c6b58-e07d-4d72-af33-ebe169402d4e-metrics-certs podName:7c5c6b58-e07d-4d72-af33-ebe169402d4e nodeName:}" failed. No retries permitted until 2026-07-08 22:15:34.998837902 +0000 UTC m=+66.266857915 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c5c6b58-e07d-4d72-af33-ebe169402d4e-metrics-certs") pod "network-metrics-daemon-jhpj6" (UID: "7c5c6b58-e07d-4d72-af33-ebe169402d4e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 22:15:03.099504 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:03.099466 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnbkg\" (UniqueName: \"kubernetes.io/projected/6a151365-c622-4361-862f-ed202f3045f7-kube-api-access-mnbkg\") pod \"network-check-target-98x2n\" (UID: \"6a151365-c622-4361-862f-ed202f3045f7\") " pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:15:03.099680 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:03.099653 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 22:15:03.099680 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:03.099678 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 22:15:03.099795 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:03.099692 2563 projected.go:196] Error preparing data for projected volume kube-api-access-mnbkg for pod openshift-network-diagnostics/network-check-target-98x2n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 22:15:03.099795 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:03.099764 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a151365-c622-4361-862f-ed202f3045f7-kube-api-access-mnbkg podName:6a151365-c622-4361-862f-ed202f3045f7 nodeName:}" failed. No retries permitted until 2026-07-08 22:15:35.099744605 +0000 UTC m=+66.367764632 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-mnbkg" (UniqueName: "kubernetes.io/projected/6a151365-c622-4361-862f-ed202f3045f7-kube-api-access-mnbkg") pod "network-check-target-98x2n" (UID: "6a151365-c622-4361-862f-ed202f3045f7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 22:15:03.312895 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:03.312848 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:15:03.313089 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:03.313006 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fffds" podUID="4a354dc2-56c0-4280-96d1-b560d70be2b2" Jul 08 22:15:04.313122 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:04.313093 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:15:04.313499 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:04.313143 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:15:04.313499 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:04.313268 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhpj6" podUID="7c5c6b58-e07d-4d72-af33-ebe169402d4e" Jul 08 22:15:04.313499 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:04.313409 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98x2n" podUID="6a151365-c622-4361-862f-ed202f3045f7" Jul 08 22:15:04.600214 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:04.600122 2563 generic.go:356] "Generic (PLEG): container finished" podID="e722d997-2e47-40ac-bb5c-48e25dde7cfd" containerID="e0f428126d2fd20fc7eb4be17374d85f384ad1443a0cd556c52f71a403cec34c" exitCode=0 Jul 08 22:15:04.600214 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:04.600194 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fbm5c" event={"ID":"e722d997-2e47-40ac-bb5c-48e25dde7cfd","Type":"ContainerDied","Data":"e0f428126d2fd20fc7eb4be17374d85f384ad1443a0cd556c52f71a403cec34c"} Jul 08 22:15:05.312852 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:05.312823 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:15:05.313036 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:05.312927 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fffds" podUID="4a354dc2-56c0-4280-96d1-b560d70be2b2" Jul 08 22:15:05.605405 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:05.605318 2563 generic.go:356] "Generic (PLEG): container finished" podID="e722d997-2e47-40ac-bb5c-48e25dde7cfd" containerID="fbec1adc28af9f0dbe3a0837d9fd5f6c5aa6861ebcb051d51b3ccf76beeb8eeb" exitCode=0 Jul 08 22:15:05.605405 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:05.605384 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fbm5c" event={"ID":"e722d997-2e47-40ac-bb5c-48e25dde7cfd","Type":"ContainerDied","Data":"fbec1adc28af9f0dbe3a0837d9fd5f6c5aa6861ebcb051d51b3ccf76beeb8eeb"} Jul 08 22:15:06.312945 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:06.312910 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:15:06.313135 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:06.312910 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:15:06.313135 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:06.313026 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhpj6" podUID="7c5c6b58-e07d-4d72-af33-ebe169402d4e" Jul 08 22:15:06.313135 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:06.313088 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98x2n" podUID="6a151365-c622-4361-862f-ed202f3045f7" Jul 08 22:15:06.611001 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:06.610916 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fbm5c" event={"ID":"e722d997-2e47-40ac-bb5c-48e25dde7cfd","Type":"ContainerStarted","Data":"315bcdf374ba6ca42d6f7224d5f1dfbabef73a4581d89823ecf26c59f8cf2ac9"} Jul 08 22:15:06.638015 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:06.636739 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-fbm5c" podStartSLOduration=9.349989561 podStartE2EDuration="37.636720883s" totalImagesPullingTime="28.286731322s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 22:14:29 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:15:06.635576328 +0000 UTC m=+37.903596359" watchObservedRunningTime="2026-07-08 22:15:06.636720883 +0000 UTC m=+37.904740913" Jul 08 22:15:07.312899 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:07.312841 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:15:07.313188 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:07.312990 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fffds" podUID="4a354dc2-56c0-4280-96d1-b560d70be2b2" Jul 08 22:15:08.313188 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:08.313148 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:15:08.313188 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:08.313169 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:15:08.313621 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:08.313265 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98x2n" podUID="6a151365-c622-4361-862f-ed202f3045f7" Jul 08 22:15:08.313621 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:08.313381 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhpj6" podUID="7c5c6b58-e07d-4d72-af33-ebe169402d4e" Jul 08 22:15:09.313417 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:09.313382 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:15:09.313789 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:09.313470 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fffds" podUID="4a354dc2-56c0-4280-96d1-b560d70be2b2" Jul 08 22:15:10.312513 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:10.312476 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:15:10.312698 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:10.312589 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98x2n" podUID="6a151365-c622-4361-862f-ed202f3045f7" Jul 08 22:15:10.312698 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:10.312644 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:15:10.312776 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:10.312738 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhpj6" podUID="7c5c6b58-e07d-4d72-af33-ebe169402d4e" Jul 08 22:15:11.312918 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:11.312875 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:15:11.313296 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:11.312989 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fffds" podUID="4a354dc2-56c0-4280-96d1-b560d70be2b2" Jul 08 22:15:12.313144 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:12.313109 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:15:12.313529 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:12.313108 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:15:12.313529 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:12.313242 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhpj6" podUID="7c5c6b58-e07d-4d72-af33-ebe169402d4e" Jul 08 22:15:12.313529 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:12.313293 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98x2n" podUID="6a151365-c622-4361-862f-ed202f3045f7" Jul 08 22:15:12.988654 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:12.988197 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fffds"] Jul 08 22:15:12.988654 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:12.988324 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:15:12.988654 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:12.988431 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fffds" podUID="4a354dc2-56c0-4280-96d1-b560d70be2b2" Jul 08 22:15:12.992424 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:12.992390 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-98x2n"] Jul 08 22:15:12.992749 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:12.992727 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:15:12.992869 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:12.992844 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98x2n" podUID="6a151365-c622-4361-862f-ed202f3045f7" Jul 08 22:15:12.993620 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:12.993264 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jhpj6"] Jul 08 22:15:12.993620 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:12.993364 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:15:12.993620 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:12.993464 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhpj6" podUID="7c5c6b58-e07d-4d72-af33-ebe169402d4e" Jul 08 22:15:14.313083 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:14.313048 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:15:14.313771 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:14.313093 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:15:14.313771 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:14.313060 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:15:14.316141 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:14.313818 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhpj6" podUID="7c5c6b58-e07d-4d72-af33-ebe169402d4e" Jul 08 22:15:14.316141 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:14.314116 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98x2n" podUID="6a151365-c622-4361-862f-ed202f3045f7" Jul 08 22:15:14.316141 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:14.314475 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fffds" podUID="4a354dc2-56c0-4280-96d1-b560d70be2b2" Jul 08 22:15:16.312983 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:16.312945 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:15:16.313375 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:16.312988 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:15:16.313375 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:16.312943 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:15:16.313375 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:16.313072 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fffds" podUID="4a354dc2-56c0-4280-96d1-b560d70be2b2" Jul 08 22:15:16.313375 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:16.313118 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98x2n" podUID="6a151365-c622-4361-862f-ed202f3045f7" Jul 08 22:15:16.313375 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:16.313237 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jhpj6" podUID="7c5c6b58-e07d-4d72-af33-ebe169402d4e" Jul 08 22:15:16.789080 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:16.789051 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-162.ec2.internal" event="NodeReady" Jul 08 22:15:16.789233 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:16.789193 2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 08 22:15:16.833049 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:16.832962 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vrwxr"] Jul 08 22:15:16.839666 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:16.839635 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-bsczr"] Jul 08 22:15:16.839815 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:16.839753 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vrwxr" Jul 08 22:15:16.842456 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:16.842408 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 08 22:15:16.842764 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:16.842749 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 08 22:15:16.842818 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:16.842805 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wwwc2\"" Jul 08 22:15:16.857956 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:16.857930 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vrwxr"] Jul 08 22:15:16.857956 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:16.857973 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bsczr"] Jul 08 22:15:16.858151 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:16.858047 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bsczr" Jul 08 22:15:16.859654 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:16.859631 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-jwrdk"] Jul 08 22:15:16.861863 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:16.861814 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 08 22:15:16.862076 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:16.862059 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 08 22:15:16.863235 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:16.863213 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 08 22:15:16.863354 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:16.863284 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 08 22:15:16.863473 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:16.863446 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-l8dh8\"" Jul 08 22:15:16.876122 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:16.876088 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jwrdk"] Jul 08 22:15:16.876257 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:16.876179 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jwrdk" Jul 08 22:15:16.878726 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:16.878705 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 08 22:15:16.878863 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:16.878759 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 08 22:15:16.878863 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:16.878772 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 08 22:15:16.878863 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:16.878816 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-n5zw5\"" Jul 08 22:15:16.996284 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:16.996244 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a9d99b14-d934-42dc-ba6a-a16effd31a85-data-volume\") pod \"insights-runtime-extractor-bsczr\" (UID: \"a9d99b14-d934-42dc-ba6a-a16effd31a85\") " pod="openshift-insights/insights-runtime-extractor-bsczr" Jul 08 22:15:16.996463 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:16.996308 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzmvh\" (UniqueName: \"kubernetes.io/projected/90dd30c2-b9af-4b4e-8f07-093761fdfeaf-kube-api-access-bzmvh\") pod \"dns-default-vrwxr\" (UID: \"90dd30c2-b9af-4b4e-8f07-093761fdfeaf\") " pod="openshift-dns/dns-default-vrwxr" Jul 08 22:15:16.996463 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:16.996329 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/90dd30c2-b9af-4b4e-8f07-093761fdfeaf-config-volume\") pod \"dns-default-vrwxr\" (UID: \"90dd30c2-b9af-4b4e-8f07-093761fdfeaf\") " pod="openshift-dns/dns-default-vrwxr" Jul 08 22:15:16.996463 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:16.996349 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rmgs\" (UniqueName: \"kubernetes.io/projected/7b839249-f20b-4b16-a761-744920da6bd0-kube-api-access-6rmgs\") pod \"ingress-canary-jwrdk\" (UID: \"7b839249-f20b-4b16-a761-744920da6bd0\") " pod="openshift-ingress-canary/ingress-canary-jwrdk" Jul 08 22:15:16.996463 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:16.996364 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90dd30c2-b9af-4b4e-8f07-093761fdfeaf-tmp-dir\") pod \"dns-default-vrwxr\" (UID: \"90dd30c2-b9af-4b4e-8f07-093761fdfeaf\") " pod="openshift-dns/dns-default-vrwxr" Jul 08 22:15:16.996463 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:16.996379 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlfcv\" (UniqueName: \"kubernetes.io/projected/a9d99b14-d934-42dc-ba6a-a16effd31a85-kube-api-access-mlfcv\") pod \"insights-runtime-extractor-bsczr\" (UID: \"a9d99b14-d934-42dc-ba6a-a16effd31a85\") " pod="openshift-insights/insights-runtime-extractor-bsczr" Jul 08 22:15:16.996656 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:16.996454 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a9d99b14-d934-42dc-ba6a-a16effd31a85-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bsczr\" (UID: \"a9d99b14-d934-42dc-ba6a-a16effd31a85\") " pod="openshift-insights/insights-runtime-extractor-bsczr" Jul 08 22:15:16.996656 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:16.996489 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b839249-f20b-4b16-a761-744920da6bd0-cert\") pod \"ingress-canary-jwrdk\" (UID: \"7b839249-f20b-4b16-a761-744920da6bd0\") " pod="openshift-ingress-canary/ingress-canary-jwrdk" Jul 08 22:15:16.996656 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:16.996507 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/90dd30c2-b9af-4b4e-8f07-093761fdfeaf-metrics-tls\") pod \"dns-default-vrwxr\" (UID: \"90dd30c2-b9af-4b4e-8f07-093761fdfeaf\") " pod="openshift-dns/dns-default-vrwxr" Jul 08 22:15:16.996656 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:16.996524 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a9d99b14-d934-42dc-ba6a-a16effd31a85-crio-socket\") pod \"insights-runtime-extractor-bsczr\" (UID: \"a9d99b14-d934-42dc-ba6a-a16effd31a85\") " pod="openshift-insights/insights-runtime-extractor-bsczr" Jul 08 22:15:16.996656 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:16.996570 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a9d99b14-d934-42dc-ba6a-a16effd31a85-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bsczr\" (UID: \"a9d99b14-d934-42dc-ba6a-a16effd31a85\") " pod="openshift-insights/insights-runtime-extractor-bsczr" Jul 08 22:15:17.097542 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:17.097467 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rmgs\" (UniqueName: \"kubernetes.io/projected/7b839249-f20b-4b16-a761-744920da6bd0-kube-api-access-6rmgs\") pod \"ingress-canary-jwrdk\" (UID: \"7b839249-f20b-4b16-a761-744920da6bd0\") " pod="openshift-ingress-canary/ingress-canary-jwrdk" Jul 08 22:15:17.097542 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:17.097505 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90dd30c2-b9af-4b4e-8f07-093761fdfeaf-tmp-dir\") pod \"dns-default-vrwxr\" (UID: \"90dd30c2-b9af-4b4e-8f07-093761fdfeaf\") " pod="openshift-dns/dns-default-vrwxr" Jul 08 22:15:17.097542 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:17.097522 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlfcv\" (UniqueName: \"kubernetes.io/projected/a9d99b14-d934-42dc-ba6a-a16effd31a85-kube-api-access-mlfcv\") pod \"insights-runtime-extractor-bsczr\" (UID: \"a9d99b14-d934-42dc-ba6a-a16effd31a85\") " pod="openshift-insights/insights-runtime-extractor-bsczr" Jul 08 22:15:17.097733 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:17.097550 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a9d99b14-d934-42dc-ba6a-a16effd31a85-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bsczr\" (UID: \"a9d99b14-d934-42dc-ba6a-a16effd31a85\") " pod="openshift-insights/insights-runtime-extractor-bsczr" Jul 08 22:15:17.097733 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:17.097588 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b839249-f20b-4b16-a761-744920da6bd0-cert\") pod \"ingress-canary-jwrdk\" (UID: \"7b839249-f20b-4b16-a761-744920da6bd0\") " pod="openshift-ingress-canary/ingress-canary-jwrdk" Jul 08 22:15:17.097733 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:17.097626 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/90dd30c2-b9af-4b4e-8f07-093761fdfeaf-metrics-tls\") pod \"dns-default-vrwxr\" (UID: \"90dd30c2-b9af-4b4e-8f07-093761fdfeaf\") " pod="openshift-dns/dns-default-vrwxr" Jul 08 22:15:17.097733 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:17.097648 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a9d99b14-d934-42dc-ba6a-a16effd31a85-crio-socket\") pod \"insights-runtime-extractor-bsczr\" (UID: \"a9d99b14-d934-42dc-ba6a-a16effd31a85\") " pod="openshift-insights/insights-runtime-extractor-bsczr" Jul 08 22:15:17.097733 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:17.097672 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a9d99b14-d934-42dc-ba6a-a16effd31a85-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bsczr\" (UID: \"a9d99b14-d934-42dc-ba6a-a16effd31a85\") " pod="openshift-insights/insights-runtime-extractor-bsczr" Jul 08 22:15:17.098042 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:17.097755 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a9d99b14-d934-42dc-ba6a-a16effd31a85-data-volume\") pod \"insights-runtime-extractor-bsczr\" (UID: \"a9d99b14-d934-42dc-ba6a-a16effd31a85\") " pod="openshift-insights/insights-runtime-extractor-bsczr" Jul 08 22:15:17.098042 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:17.097839 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzmvh\" (UniqueName: \"kubernetes.io/projected/90dd30c2-b9af-4b4e-8f07-093761fdfeaf-kube-api-access-bzmvh\") pod \"dns-default-vrwxr\" (UID: \"90dd30c2-b9af-4b4e-8f07-093761fdfeaf\") " pod="openshift-dns/dns-default-vrwxr" Jul 08 22:15:17.098042 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:17.097867 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/90dd30c2-b9af-4b4e-8f07-093761fdfeaf-config-volume\") pod \"dns-default-vrwxr\" (UID: \"90dd30c2-b9af-4b4e-8f07-093761fdfeaf\") " pod="openshift-dns/dns-default-vrwxr" Jul 08 22:15:17.098042 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:17.097963 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a9d99b14-d934-42dc-ba6a-a16effd31a85-crio-socket\") pod \"insights-runtime-extractor-bsczr\" (UID: \"a9d99b14-d934-42dc-ba6a-a16effd31a85\") " pod="openshift-insights/insights-runtime-extractor-bsczr" Jul 08 22:15:17.098042 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:17.097979 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90dd30c2-b9af-4b4e-8f07-093761fdfeaf-tmp-dir\") pod \"dns-default-vrwxr\" (UID: \"90dd30c2-b9af-4b4e-8f07-093761fdfeaf\") " pod="openshift-dns/dns-default-vrwxr" Jul 08 22:15:17.098287 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:17.098214 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a9d99b14-d934-42dc-ba6a-a16effd31a85-data-volume\") pod \"insights-runtime-extractor-bsczr\" (UID: \"a9d99b14-d934-42dc-ba6a-a16effd31a85\") " pod="openshift-insights/insights-runtime-extractor-bsczr" Jul 08 22:15:17.098396 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:17.098378 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/90dd30c2-b9af-4b4e-8f07-093761fdfeaf-config-volume\") pod \"dns-default-vrwxr\" (UID: \"90dd30c2-b9af-4b4e-8f07-093761fdfeaf\") " pod="openshift-dns/dns-default-vrwxr" Jul 08 22:15:17.098466 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:17.098446 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a9d99b14-d934-42dc-ba6a-a16effd31a85-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bsczr\" (UID: \"a9d99b14-d934-42dc-ba6a-a16effd31a85\") " pod="openshift-insights/insights-runtime-extractor-bsczr" Jul 08 22:15:17.101974 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:17.101947 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a9d99b14-d934-42dc-ba6a-a16effd31a85-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bsczr\" (UID: \"a9d99b14-d934-42dc-ba6a-a16effd31a85\") " pod="openshift-insights/insights-runtime-extractor-bsczr" Jul 08 22:15:17.101974 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:17.101961 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/90dd30c2-b9af-4b4e-8f07-093761fdfeaf-metrics-tls\") pod \"dns-default-vrwxr\" (UID: \"90dd30c2-b9af-4b4e-8f07-093761fdfeaf\") " pod="openshift-dns/dns-default-vrwxr" Jul 08 22:15:17.102135 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:17.101985 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b839249-f20b-4b16-a761-744920da6bd0-cert\") pod \"ingress-canary-jwrdk\" (UID: \"7b839249-f20b-4b16-a761-744920da6bd0\") " pod="openshift-ingress-canary/ingress-canary-jwrdk" Jul 08 22:15:17.106177 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:17.106149 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzmvh\" (UniqueName: \"kubernetes.io/projected/90dd30c2-b9af-4b4e-8f07-093761fdfeaf-kube-api-access-bzmvh\") pod \"dns-default-vrwxr\" (UID: \"90dd30c2-b9af-4b4e-8f07-093761fdfeaf\") " pod="openshift-dns/dns-default-vrwxr" Jul 08 22:15:17.106296 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:17.106204 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rmgs\" (UniqueName: \"kubernetes.io/projected/7b839249-f20b-4b16-a761-744920da6bd0-kube-api-access-6rmgs\") pod \"ingress-canary-jwrdk\" (UID: \"7b839249-f20b-4b16-a761-744920da6bd0\") " pod="openshift-ingress-canary/ingress-canary-jwrdk" Jul 08 22:15:17.106296 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:17.106235 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlfcv\" (UniqueName: \"kubernetes.io/projected/a9d99b14-d934-42dc-ba6a-a16effd31a85-kube-api-access-mlfcv\") pod \"insights-runtime-extractor-bsczr\" (UID: \"a9d99b14-d934-42dc-ba6a-a16effd31a85\") " pod="openshift-insights/insights-runtime-extractor-bsczr" Jul 08 22:15:17.150134 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:17.150097 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vrwxr" Jul 08 22:15:17.167276 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:17.167242 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bsczr" Jul 08 22:15:17.187200 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:17.186982 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jwrdk" Jul 08 22:15:17.366237 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:17.366152 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jwrdk"] Jul 08 22:15:17.370623 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:15:17.370593 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b839249_f20b_4b16_a761_744920da6bd0.slice/crio-deefeae69806b08f95b465feac33a9cea0a63db7b501fe31bf9c806693dd353e WatchSource:0}: Error finding container deefeae69806b08f95b465feac33a9cea0a63db7b501fe31bf9c806693dd353e: Status 404 returned error can't find the container with id deefeae69806b08f95b465feac33a9cea0a63db7b501fe31bf9c806693dd353e Jul 08 22:15:17.383112 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:17.383085 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bsczr"] Jul 08 22:15:17.385981 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:15:17.385954 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9d99b14_d934_42dc_ba6a_a16effd31a85.slice/crio-46e29b19a2cee24d9e7f14454af1bfe71d390af09bbc6eb072d07b5992091882 WatchSource:0}: Error finding container 46e29b19a2cee24d9e7f14454af1bfe71d390af09bbc6eb072d07b5992091882: Status 404 returned error can't find the container with id 46e29b19a2cee24d9e7f14454af1bfe71d390af09bbc6eb072d07b5992091882 Jul 08 22:15:17.394742 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:17.394717 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vrwxr"] Jul 08 22:15:17.397485 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:15:17.397457 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90dd30c2_b9af_4b4e_8f07_093761fdfeaf.slice/crio-e12d942b121f3e9ec66eb1a42a6a3cc7acbcd2c5b05cf073e3f39f97739e9401 WatchSource:0}: Error finding container e12d942b121f3e9ec66eb1a42a6a3cc7acbcd2c5b05cf073e3f39f97739e9401: Status 404 returned error can't find the container with id e12d942b121f3e9ec66eb1a42a6a3cc7acbcd2c5b05cf073e3f39f97739e9401 Jul 08 22:15:17.639091 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:17.638996 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bsczr" event={"ID":"a9d99b14-d934-42dc-ba6a-a16effd31a85","Type":"ContainerStarted","Data":"46e29b19a2cee24d9e7f14454af1bfe71d390af09bbc6eb072d07b5992091882"} Jul 08 22:15:17.640092 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:17.640058 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jwrdk" event={"ID":"7b839249-f20b-4b16-a761-744920da6bd0","Type":"ContainerStarted","Data":"deefeae69806b08f95b465feac33a9cea0a63db7b501fe31bf9c806693dd353e"} Jul 08 22:15:17.641110 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:17.641089 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vrwxr" event={"ID":"90dd30c2-b9af-4b4e-8f07-093761fdfeaf","Type":"ContainerStarted","Data":"e12d942b121f3e9ec66eb1a42a6a3cc7acbcd2c5b05cf073e3f39f97739e9401"} Jul 08 22:15:18.312763 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.312554 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:15:18.312763 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.312557 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:15:18.312763 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.312705 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:15:18.318375 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.316602 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-4rrdh\"" Jul 08 22:15:18.318375 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.316934 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 22:15:18.318375 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.317153 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bwnr2\"" Jul 08 22:15:18.318375 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.317367 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 22:15:18.318375 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.317670 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 22:15:18.318375 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.317906 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 22:15:18.341313 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.339084 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-658dc76d8d-lx2hh"] Jul 08 22:15:18.353676 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.352175 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-658dc76d8d-lx2hh" Jul 08 22:15:18.355493 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.355464 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-6jpc4\"" Jul 08 22:15:18.355721 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.355701 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 08 22:15:18.355983 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.355965 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 08 22:15:18.356193 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.356176 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 08 22:15:18.356524 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.356505 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 08 22:15:18.357075 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.356783 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 08 22:15:18.357075 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.357005 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 08 22:15:18.357251 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.357231 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 08 22:15:18.358543 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.358524 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-658dc76d8d-lx2hh"] Jul 08 22:15:18.510301 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.510266 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/28e04205-7edb-4465-928d-ef26a27ae65c-console-config\") pod \"console-658dc76d8d-lx2hh\" (UID: \"28e04205-7edb-4465-928d-ef26a27ae65c\") " pod="openshift-console/console-658dc76d8d-lx2hh" Jul 08 22:15:18.510684 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.510310 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/28e04205-7edb-4465-928d-ef26a27ae65c-service-ca\") pod \"console-658dc76d8d-lx2hh\" (UID: \"28e04205-7edb-4465-928d-ef26a27ae65c\") " pod="openshift-console/console-658dc76d8d-lx2hh" Jul 08 22:15:18.510684 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.510364 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwnhh\" (UniqueName: \"kubernetes.io/projected/28e04205-7edb-4465-928d-ef26a27ae65c-kube-api-access-nwnhh\") pod \"console-658dc76d8d-lx2hh\" (UID: \"28e04205-7edb-4465-928d-ef26a27ae65c\") " pod="openshift-console/console-658dc76d8d-lx2hh" Jul 08 22:15:18.510684 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.510430 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/28e04205-7edb-4465-928d-ef26a27ae65c-oauth-serving-cert\") pod \"console-658dc76d8d-lx2hh\" (UID: \"28e04205-7edb-4465-928d-ef26a27ae65c\") " pod="openshift-console/console-658dc76d8d-lx2hh" Jul 08 22:15:18.510684 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.510456 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/28e04205-7edb-4465-928d-ef26a27ae65c-console-serving-cert\") pod \"console-658dc76d8d-lx2hh\" (UID: \"28e04205-7edb-4465-928d-ef26a27ae65c\") " pod="openshift-console/console-658dc76d8d-lx2hh" Jul 08 22:15:18.510684 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.510492 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/28e04205-7edb-4465-928d-ef26a27ae65c-console-oauth-config\") pod \"console-658dc76d8d-lx2hh\" (UID: \"28e04205-7edb-4465-928d-ef26a27ae65c\") " pod="openshift-console/console-658dc76d8d-lx2hh" Jul 08 22:15:18.611673 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.611582 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/28e04205-7edb-4465-928d-ef26a27ae65c-oauth-serving-cert\") pod \"console-658dc76d8d-lx2hh\" (UID: \"28e04205-7edb-4465-928d-ef26a27ae65c\") " pod="openshift-console/console-658dc76d8d-lx2hh" Jul 08 22:15:18.611673 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.611629 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/28e04205-7edb-4465-928d-ef26a27ae65c-console-serving-cert\") pod \"console-658dc76d8d-lx2hh\" (UID: \"28e04205-7edb-4465-928d-ef26a27ae65c\") " pod="openshift-console/console-658dc76d8d-lx2hh" Jul 08 22:15:18.611673 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.611658 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/28e04205-7edb-4465-928d-ef26a27ae65c-console-oauth-config\") pod \"console-658dc76d8d-lx2hh\" (UID: \"28e04205-7edb-4465-928d-ef26a27ae65c\") " pod="openshift-console/console-658dc76d8d-lx2hh" Jul 08 22:15:18.611980 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.611697 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/28e04205-7edb-4465-928d-ef26a27ae65c-console-config\") pod \"console-658dc76d8d-lx2hh\" (UID: \"28e04205-7edb-4465-928d-ef26a27ae65c\") " pod="openshift-console/console-658dc76d8d-lx2hh" Jul 08 22:15:18.611980 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.611723 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/28e04205-7edb-4465-928d-ef26a27ae65c-service-ca\") pod \"console-658dc76d8d-lx2hh\" (UID: \"28e04205-7edb-4465-928d-ef26a27ae65c\") " pod="openshift-console/console-658dc76d8d-lx2hh" Jul 08 22:15:18.611980 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.611755 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwnhh\" (UniqueName: \"kubernetes.io/projected/28e04205-7edb-4465-928d-ef26a27ae65c-kube-api-access-nwnhh\") pod \"console-658dc76d8d-lx2hh\" (UID: \"28e04205-7edb-4465-928d-ef26a27ae65c\") " pod="openshift-console/console-658dc76d8d-lx2hh" Jul 08 22:15:18.612496 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.612464 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/28e04205-7edb-4465-928d-ef26a27ae65c-oauth-serving-cert\") pod \"console-658dc76d8d-lx2hh\" (UID: \"28e04205-7edb-4465-928d-ef26a27ae65c\") " pod="openshift-console/console-658dc76d8d-lx2hh" Jul 08 22:15:18.612946 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.612920 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/28e04205-7edb-4465-928d-ef26a27ae65c-service-ca\") pod \"console-658dc76d8d-lx2hh\" (UID: \"28e04205-7edb-4465-928d-ef26a27ae65c\") " pod="openshift-console/console-658dc76d8d-lx2hh" Jul 08 22:15:18.613037 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.612950 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/28e04205-7edb-4465-928d-ef26a27ae65c-console-config\") pod \"console-658dc76d8d-lx2hh\" (UID: \"28e04205-7edb-4465-928d-ef26a27ae65c\") " pod="openshift-console/console-658dc76d8d-lx2hh" Jul 08 22:15:18.615423 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.615400 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/28e04205-7edb-4465-928d-ef26a27ae65c-console-oauth-config\") pod \"console-658dc76d8d-lx2hh\" (UID: \"28e04205-7edb-4465-928d-ef26a27ae65c\") " pod="openshift-console/console-658dc76d8d-lx2hh" Jul 08 22:15:18.619389 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.619328 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/28e04205-7edb-4465-928d-ef26a27ae65c-console-serving-cert\") pod \"console-658dc76d8d-lx2hh\" (UID: \"28e04205-7edb-4465-928d-ef26a27ae65c\") " pod="openshift-console/console-658dc76d8d-lx2hh" Jul 08 22:15:18.627104 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.627073 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwnhh\" (UniqueName: \"kubernetes.io/projected/28e04205-7edb-4465-928d-ef26a27ae65c-kube-api-access-nwnhh\") pod \"console-658dc76d8d-lx2hh\" (UID: \"28e04205-7edb-4465-928d-ef26a27ae65c\") " pod="openshift-console/console-658dc76d8d-lx2hh" Jul 08 22:15:18.669944 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.669906 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-658dc76d8d-lx2hh" Jul 08 22:15:18.862056 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:18.861971 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-658dc76d8d-lx2hh"] Jul 08 22:15:18.866120 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:15:18.866035 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28e04205_7edb_4465_928d_ef26a27ae65c.slice/crio-fba51e140bc6f62468f2d342a13b7b8aee547998e98a99e88a140248a53f270e WatchSource:0}: Error finding container fba51e140bc6f62468f2d342a13b7b8aee547998e98a99e88a140248a53f270e: Status 404 returned error can't find the container with id fba51e140bc6f62468f2d342a13b7b8aee547998e98a99e88a140248a53f270e Jul 08 22:15:19.648452 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:19.648413 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-658dc76d8d-lx2hh" event={"ID":"28e04205-7edb-4465-928d-ef26a27ae65c","Type":"ContainerStarted","Data":"fba51e140bc6f62468f2d342a13b7b8aee547998e98a99e88a140248a53f270e"} Jul 08 22:15:21.006559 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:21.006519 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-w2c67"] Jul 08 22:15:21.019917 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:21.019807 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w2c67" Jul 08 22:15:21.020473 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:21.020044 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-w2c67"] Jul 08 22:15:21.023058 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:21.023030 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 08 22:15:21.023458 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:21.023427 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 08 22:15:21.024733 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:21.024522 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 08 22:15:21.024733 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:21.024629 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 08 22:15:21.024733 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:21.024632 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 08 22:15:21.025372 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:21.025204 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-tgqxh\"" Jul 08 22:15:21.031970 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:21.031948 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/67767774-0fce-4cea-b8ba-3906c5af3293-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-w2c67\" (UID: \"67767774-0fce-4cea-b8ba-3906c5af3293\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w2c67" Jul 08 22:15:21.032069 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:21.032004 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67767774-0fce-4cea-b8ba-3906c5af3293-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-w2c67\" (UID: \"67767774-0fce-4cea-b8ba-3906c5af3293\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w2c67" Jul 08 22:15:21.032151 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:21.032126 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55cbc\" (UniqueName: \"kubernetes.io/projected/67767774-0fce-4cea-b8ba-3906c5af3293-kube-api-access-55cbc\") pod \"prometheus-operator-567f7b8b7c-w2c67\" (UID: \"67767774-0fce-4cea-b8ba-3906c5af3293\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w2c67" Jul 08 22:15:21.032195 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:21.032180 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67767774-0fce-4cea-b8ba-3906c5af3293-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-w2c67\" (UID: \"67767774-0fce-4cea-b8ba-3906c5af3293\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w2c67" Jul 08 22:15:21.133468 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:21.133424 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55cbc\" (UniqueName: \"kubernetes.io/projected/67767774-0fce-4cea-b8ba-3906c5af3293-kube-api-access-55cbc\") pod \"prometheus-operator-567f7b8b7c-w2c67\" (UID: \"67767774-0fce-4cea-b8ba-3906c5af3293\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w2c67" Jul 08 22:15:21.133468 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:21.133484 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67767774-0fce-4cea-b8ba-3906c5af3293-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-w2c67\" (UID: \"67767774-0fce-4cea-b8ba-3906c5af3293\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w2c67" Jul 08 22:15:21.133726 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:21.133540 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/67767774-0fce-4cea-b8ba-3906c5af3293-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-w2c67\" (UID: \"67767774-0fce-4cea-b8ba-3906c5af3293\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w2c67" Jul 08 22:15:21.133726 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:21.133583 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67767774-0fce-4cea-b8ba-3906c5af3293-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-w2c67\" (UID: \"67767774-0fce-4cea-b8ba-3906c5af3293\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w2c67" Jul 08 22:15:21.133828 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:21.133727 2563 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 08 22:15:21.133828 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:21.133795 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67767774-0fce-4cea-b8ba-3906c5af3293-prometheus-operator-tls podName:67767774-0fce-4cea-b8ba-3906c5af3293 nodeName:}" failed. No retries permitted until 2026-07-08 22:15:21.633778336 +0000 UTC m=+52.901798341 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/67767774-0fce-4cea-b8ba-3906c5af3293-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-w2c67" (UID: "67767774-0fce-4cea-b8ba-3906c5af3293") : secret "prometheus-operator-tls" not found Jul 08 22:15:21.134412 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:21.134385 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67767774-0fce-4cea-b8ba-3906c5af3293-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-w2c67\" (UID: \"67767774-0fce-4cea-b8ba-3906c5af3293\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w2c67" Jul 08 22:15:21.136116 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:21.136091 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67767774-0fce-4cea-b8ba-3906c5af3293-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-w2c67\" (UID: \"67767774-0fce-4cea-b8ba-3906c5af3293\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w2c67" Jul 08 22:15:21.146022 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:21.145996 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55cbc\" (UniqueName: \"kubernetes.io/projected/67767774-0fce-4cea-b8ba-3906c5af3293-kube-api-access-55cbc\") pod \"prometheus-operator-567f7b8b7c-w2c67\" (UID: \"67767774-0fce-4cea-b8ba-3906c5af3293\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w2c67" Jul 08 22:15:21.637175 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:21.637140 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/67767774-0fce-4cea-b8ba-3906c5af3293-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-w2c67\" (UID: \"67767774-0fce-4cea-b8ba-3906c5af3293\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w2c67" Jul 08 22:15:21.639489 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:21.639463 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/67767774-0fce-4cea-b8ba-3906c5af3293-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-w2c67\" (UID: \"67767774-0fce-4cea-b8ba-3906c5af3293\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w2c67" Jul 08 22:15:21.931661 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:21.931575 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w2c67" Jul 08 22:15:24.327231 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:24.327196 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c5f6bd69f-2l4z7"] Jul 08 22:15:24.409179 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:24.409138 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c5f6bd69f-2l4z7"] Jul 08 22:15:24.409348 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:24.409260 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5f6bd69f-2l4z7" Jul 08 22:15:24.418240 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:24.418217 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 08 22:15:24.461307 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:24.461268 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5d62c766-320a-4fea-ac5a-23db8702e521-oauth-serving-cert\") pod \"console-5c5f6bd69f-2l4z7\" (UID: \"5d62c766-320a-4fea-ac5a-23db8702e521\") " pod="openshift-console/console-5c5f6bd69f-2l4z7" Jul 08 22:15:24.461471 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:24.461320 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5d62c766-320a-4fea-ac5a-23db8702e521-console-serving-cert\") pod \"console-5c5f6bd69f-2l4z7\" (UID: \"5d62c766-320a-4fea-ac5a-23db8702e521\") " pod="openshift-console/console-5c5f6bd69f-2l4z7" Jul 08 22:15:24.461471 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:24.461344 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d62c766-320a-4fea-ac5a-23db8702e521-trusted-ca-bundle\") pod \"console-5c5f6bd69f-2l4z7\" (UID: \"5d62c766-320a-4fea-ac5a-23db8702e521\") " pod="openshift-console/console-5c5f6bd69f-2l4z7" Jul 08 22:15:24.461471 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:24.461393 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5d62c766-320a-4fea-ac5a-23db8702e521-console-oauth-config\") pod \"console-5c5f6bd69f-2l4z7\" (UID: \"5d62c766-320a-4fea-ac5a-23db8702e521\") " pod="openshift-console/console-5c5f6bd69f-2l4z7" Jul 08 22:15:24.461471 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:24.461433 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5d62c766-320a-4fea-ac5a-23db8702e521-service-ca\") pod \"console-5c5f6bd69f-2l4z7\" (UID: \"5d62c766-320a-4fea-ac5a-23db8702e521\") " pod="openshift-console/console-5c5f6bd69f-2l4z7" Jul 08 22:15:24.461601 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:24.461477 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5d62c766-320a-4fea-ac5a-23db8702e521-console-config\") pod \"console-5c5f6bd69f-2l4z7\" (UID: \"5d62c766-320a-4fea-ac5a-23db8702e521\") " pod="openshift-console/console-5c5f6bd69f-2l4z7" Jul 08 22:15:24.461601 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:24.461500 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgvzk\" (UniqueName: \"kubernetes.io/projected/5d62c766-320a-4fea-ac5a-23db8702e521-kube-api-access-bgvzk\") pod \"console-5c5f6bd69f-2l4z7\" (UID: \"5d62c766-320a-4fea-ac5a-23db8702e521\") " pod="openshift-console/console-5c5f6bd69f-2l4z7" Jul 08 22:15:24.562791 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:24.562753 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d62c766-320a-4fea-ac5a-23db8702e521-trusted-ca-bundle\") pod \"console-5c5f6bd69f-2l4z7\" (UID: \"5d62c766-320a-4fea-ac5a-23db8702e521\") " pod="openshift-console/console-5c5f6bd69f-2l4z7" Jul 08 22:15:24.562791 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:24.562797 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5d62c766-320a-4fea-ac5a-23db8702e521-console-oauth-config\") pod \"console-5c5f6bd69f-2l4z7\" (UID: \"5d62c766-320a-4fea-ac5a-23db8702e521\") " pod="openshift-console/console-5c5f6bd69f-2l4z7" Jul 08 22:15:24.563046 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:24.562824 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5d62c766-320a-4fea-ac5a-23db8702e521-service-ca\") pod \"console-5c5f6bd69f-2l4z7\" (UID: \"5d62c766-320a-4fea-ac5a-23db8702e521\") " pod="openshift-console/console-5c5f6bd69f-2l4z7" Jul 08 22:15:24.563046 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:24.562858 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5d62c766-320a-4fea-ac5a-23db8702e521-console-config\") pod \"console-5c5f6bd69f-2l4z7\" (UID: \"5d62c766-320a-4fea-ac5a-23db8702e521\") " pod="openshift-console/console-5c5f6bd69f-2l4z7" Jul 08 22:15:24.563046 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:24.562905 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgvzk\" (UniqueName: \"kubernetes.io/projected/5d62c766-320a-4fea-ac5a-23db8702e521-kube-api-access-bgvzk\") pod \"console-5c5f6bd69f-2l4z7\" (UID: \"5d62c766-320a-4fea-ac5a-23db8702e521\") " pod="openshift-console/console-5c5f6bd69f-2l4z7" Jul 08 22:15:24.563046 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:24.562955 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5d62c766-320a-4fea-ac5a-23db8702e521-oauth-serving-cert\") pod \"console-5c5f6bd69f-2l4z7\" (UID: \"5d62c766-320a-4fea-ac5a-23db8702e521\") " pod="openshift-console/console-5c5f6bd69f-2l4z7" Jul 08 22:15:24.563046 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:24.562990 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5d62c766-320a-4fea-ac5a-23db8702e521-console-serving-cert\") pod \"console-5c5f6bd69f-2l4z7\" (UID: \"5d62c766-320a-4fea-ac5a-23db8702e521\") " pod="openshift-console/console-5c5f6bd69f-2l4z7" Jul 08 22:15:24.563718 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:24.563689 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5d62c766-320a-4fea-ac5a-23db8702e521-console-config\") pod \"console-5c5f6bd69f-2l4z7\" (UID: \"5d62c766-320a-4fea-ac5a-23db8702e521\") " pod="openshift-console/console-5c5f6bd69f-2l4z7" Jul 08 22:15:24.563823 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:24.563774 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5d62c766-320a-4fea-ac5a-23db8702e521-oauth-serving-cert\") pod \"console-5c5f6bd69f-2l4z7\" (UID: \"5d62c766-320a-4fea-ac5a-23db8702e521\") " pod="openshift-console/console-5c5f6bd69f-2l4z7" Jul 08 22:15:24.563823 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:24.563779 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5d62c766-320a-4fea-ac5a-23db8702e521-service-ca\") pod \"console-5c5f6bd69f-2l4z7\" (UID: \"5d62c766-320a-4fea-ac5a-23db8702e521\") " pod="openshift-console/console-5c5f6bd69f-2l4z7" Jul 08 22:15:24.563823 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:24.563780 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d62c766-320a-4fea-ac5a-23db8702e521-trusted-ca-bundle\") pod \"console-5c5f6bd69f-2l4z7\" (UID: \"5d62c766-320a-4fea-ac5a-23db8702e521\") " pod="openshift-console/console-5c5f6bd69f-2l4z7" Jul 08 22:15:24.565683 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:24.565652 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5d62c766-320a-4fea-ac5a-23db8702e521-console-oauth-config\") pod \"console-5c5f6bd69f-2l4z7\" (UID: \"5d62c766-320a-4fea-ac5a-23db8702e521\") " pod="openshift-console/console-5c5f6bd69f-2l4z7" Jul 08 22:15:24.565744 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:24.565708 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5d62c766-320a-4fea-ac5a-23db8702e521-console-serving-cert\") pod \"console-5c5f6bd69f-2l4z7\" (UID: \"5d62c766-320a-4fea-ac5a-23db8702e521\") " pod="openshift-console/console-5c5f6bd69f-2l4z7" Jul 08 22:15:24.572021 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:24.571996 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgvzk\" (UniqueName: \"kubernetes.io/projected/5d62c766-320a-4fea-ac5a-23db8702e521-kube-api-access-bgvzk\") pod \"console-5c5f6bd69f-2l4z7\" (UID: \"5d62c766-320a-4fea-ac5a-23db8702e521\") " pod="openshift-console/console-5c5f6bd69f-2l4z7" Jul 08 22:15:24.719719 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:24.719684 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5f6bd69f-2l4z7" Jul 08 22:15:25.220070 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:25.220039 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c5f6bd69f-2l4z7"] Jul 08 22:15:25.223590 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:15:25.223555 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d62c766_320a_4fea_ac5a_23db8702e521.slice/crio-63ddf2cdc2f2ad42cdf12002f3bae7e3e54aedcf2e38cc459528dcfcf6e61cab WatchSource:0}: Error finding container 63ddf2cdc2f2ad42cdf12002f3bae7e3e54aedcf2e38cc459528dcfcf6e61cab: Status 404 returned error can't find the container with id 63ddf2cdc2f2ad42cdf12002f3bae7e3e54aedcf2e38cc459528dcfcf6e61cab Jul 08 22:15:25.232764 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:25.232739 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-w2c67"] Jul 08 22:15:25.236502 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:15:25.236478 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67767774_0fce_4cea_b8ba_3906c5af3293.slice/crio-3ad892881e0b87faa620b0495526f90707bdae3bf1c0707e7b8b6e804a329e38 WatchSource:0}: Error finding container 3ad892881e0b87faa620b0495526f90707bdae3bf1c0707e7b8b6e804a329e38: Status 404 returned error can't find the container with id 3ad892881e0b87faa620b0495526f90707bdae3bf1c0707e7b8b6e804a329e38 Jul 08 22:15:25.669982 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:25.669832 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w2c67" event={"ID":"67767774-0fce-4cea-b8ba-3906c5af3293","Type":"ContainerStarted","Data":"3ad892881e0b87faa620b0495526f90707bdae3bf1c0707e7b8b6e804a329e38"} Jul 08 22:15:25.671376 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:25.671346 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bsczr" event={"ID":"a9d99b14-d934-42dc-ba6a-a16effd31a85","Type":"ContainerStarted","Data":"6fa1610738e46ea4f40d973db59a2be7914d421a3024b748009e220847b1af4d"} Jul 08 22:15:25.672915 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:25.672871 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-658dc76d8d-lx2hh" event={"ID":"28e04205-7edb-4465-928d-ef26a27ae65c","Type":"ContainerStarted","Data":"f33b42f41e37533e0861ef2d7ba3ba63bb138374d9b60c37b98087a0a9679c49"} Jul 08 22:15:25.674416 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:25.674396 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5f6bd69f-2l4z7" event={"ID":"5d62c766-320a-4fea-ac5a-23db8702e521","Type":"ContainerStarted","Data":"0faa4e6d1326ebaa4a099fa9d99917ba21fce14ceba5f75be33e005aa2383ec6"} Jul 08 22:15:25.674509 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:25.674433 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5f6bd69f-2l4z7" event={"ID":"5d62c766-320a-4fea-ac5a-23db8702e521","Type":"ContainerStarted","Data":"63ddf2cdc2f2ad42cdf12002f3bae7e3e54aedcf2e38cc459528dcfcf6e61cab"} Jul 08 22:15:25.675822 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:25.675800 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jwrdk" event={"ID":"7b839249-f20b-4b16-a761-744920da6bd0","Type":"ContainerStarted","Data":"0fa1545d8df77e5dd2c9b2f7cf74dc750fd70ea568a65698dec134f7d2750e1c"} Jul 08 22:15:25.677466 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:25.677444 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vrwxr" event={"ID":"90dd30c2-b9af-4b4e-8f07-093761fdfeaf","Type":"ContainerStarted","Data":"f8cebf5889b8c2aa7373a2a4f76014a4318c8c098f45aa01293c397553adc39e"} Jul 08 22:15:25.677535 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:25.677489 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vrwxr" event={"ID":"90dd30c2-b9af-4b4e-8f07-093761fdfeaf","Type":"ContainerStarted","Data":"13b80e0e867547b712b17ed4cf5b9952cca5842a296cc65eabc9dc203eed36bc"} Jul 08 22:15:25.677535 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:25.677525 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vrwxr" Jul 08 22:15:25.690828 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:25.690777 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-658dc76d8d-lx2hh" podStartSLOduration=1.4852174919999999 podStartE2EDuration="7.690758821s" totalImagesPullingTime="6.205541329s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 22:15:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:15:25.69045998 +0000 UTC m=+56.958480012" watchObservedRunningTime="2026-07-08 22:15:25.690758821 +0000 UTC m=+56.958778843" Jul 08 22:15:25.707709 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:25.707648 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vrwxr" podStartSLOduration=2.035825904 podStartE2EDuration="9.707631523s" totalImagesPullingTime="7.671805619s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 22:15:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:15:25.706814131 +0000 UTC m=+56.974834159" watchObservedRunningTime="2026-07-08 22:15:25.707631523 +0000 UTC m=+56.975651612" Jul 08 22:15:25.721411 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:25.721356 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-jwrdk" podStartSLOduration=2.023360361 podStartE2EDuration="9.721337898s" totalImagesPullingTime="7.697977537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 22:15:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:15:25.721262571 +0000 UTC m=+56.989282600" watchObservedRunningTime="2026-07-08 22:15:25.721337898 +0000 UTC m=+56.989357926" Jul 08 22:15:25.738524 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:25.738464 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5c5f6bd69f-2l4z7" podStartSLOduration=1.738416704 podStartE2EDuration="1.738416704s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 22:15:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:15:25.73750473 +0000 UTC m=+57.005524767" watchObservedRunningTime="2026-07-08 22:15:25.738416704 +0000 UTC m=+57.006436733" Jul 08 22:15:26.684107 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:26.684066 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bsczr" event={"ID":"a9d99b14-d934-42dc-ba6a-a16effd31a85","Type":"ContainerStarted","Data":"c9debc91f9822aa0d5e4eefaeb6e122d8dfb1a065bbca0f3b984b52c3ac8fa4a"} Jul 08 22:15:27.689765 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:27.689732 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w2c67" event={"ID":"67767774-0fce-4cea-b8ba-3906c5af3293","Type":"ContainerStarted","Data":"e3a3cdd6fe8cad3cd478df915cc090708d89b293a6cf35642eb60d0791e67cb9"} Jul 08 22:15:27.690200 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:27.689784 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w2c67" event={"ID":"67767774-0fce-4cea-b8ba-3906c5af3293","Type":"ContainerStarted","Data":"05c556c74249e1fe14a7e6880f83987759c1383a98ec27ae49ddfbc62e65b2ab"} Jul 08 22:15:27.710165 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:27.710108 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w2c67" podStartSLOduration=5.807226262 podStartE2EDuration="7.710088897s" totalImagesPullingTime="1.902862635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 22:15:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:15:27.708366226 +0000 UTC m=+58.976386257" watchObservedRunningTime="2026-07-08 22:15:27.710088897 +0000 UTC m=+58.978108924" Jul 08 22:15:28.670385 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:28.670339 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-658dc76d8d-lx2hh" Jul 08 22:15:28.670558 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:28.670433 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-658dc76d8d-lx2hh" Jul 08 22:15:28.671946 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:28.671915 2563 patch_prober.go:28] interesting pod/console-658dc76d8d-lx2hh container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused" start-of-body= Jul 08 22:15:28.672042 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:28.671988 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-658dc76d8d-lx2hh" podUID="28e04205-7edb-4465-928d-ef26a27ae65c" containerName="console" probeResult="failure" output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused" Jul 08 22:15:28.696087 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:28.696048 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bsczr" event={"ID":"a9d99b14-d934-42dc-ba6a-a16effd31a85","Type":"ContainerStarted","Data":"5875274a2f79a41650ff923aa22feb6afbdee109e9c632a2a0601fa88e4ea758"} Jul 08 22:15:28.714970 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:28.714915 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-bsczr" podStartSLOduration=1.603555801 podStartE2EDuration="12.714879808s" totalImagesPullingTime="11.111324007s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 22:15:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:15:28.712958756 +0000 UTC m=+59.980978786" watchObservedRunningTime="2026-07-08 22:15:28.714879808 +0000 UTC m=+59.982899835" Jul 08 22:15:29.342783 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.342751 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-jcw5n"] Jul 08 22:15:29.346176 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.346158 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jcw5n" Jul 08 22:15:29.348806 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.348784 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 08 22:15:29.349168 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.349148 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 08 22:15:29.349405 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.349391 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-2d6j2\"" Jul 08 22:15:29.361660 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.361628 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-jcw5n"] Jul 08 22:15:29.362387 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.362364 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kqq85"] Jul 08 22:15:29.364625 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.364605 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kqq85" Jul 08 22:15:29.368292 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.368267 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 08 22:15:29.368292 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.368280 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 08 22:15:29.368292 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.368293 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-bvbfd\"" Jul 08 22:15:29.368510 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.368377 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 08 22:15:29.379816 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.379522 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kqq85"] Jul 08 22:15:29.389649 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.389621 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-46rlm"] Jul 08 22:15:29.391983 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.391959 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-46rlm" Jul 08 22:15:29.394610 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.394588 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 08 22:15:29.394762 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.394615 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 08 22:15:29.394762 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.394660 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 08 22:15:29.394762 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.394740 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-ml8tb\"" Jul 08 22:15:29.403528 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.403500 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9ba2c75b-169e-4f62-85a4-34171290a5bf-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-kqq85\" (UID: \"9ba2c75b-169e-4f62-85a4-34171290a5bf\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kqq85" Jul 08 22:15:29.403672 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.403535 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9ba2c75b-169e-4f62-85a4-34171290a5bf-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-kqq85\" (UID: \"9ba2c75b-169e-4f62-85a4-34171290a5bf\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kqq85" Jul 08 22:15:29.403672 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.403556 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5085739c-ac1f-4818-9a2a-065a2bc6cab1-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-jcw5n\" (UID: \"5085739c-ac1f-4818-9a2a-065a2bc6cab1\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jcw5n" Jul 08 22:15:29.403672 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.403612 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5085739c-ac1f-4818-9a2a-065a2bc6cab1-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-jcw5n\" (UID: \"5085739c-ac1f-4818-9a2a-065a2bc6cab1\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jcw5n" Jul 08 22:15:29.403672 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.403647 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5085739c-ac1f-4818-9a2a-065a2bc6cab1-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-jcw5n\" (UID: \"5085739c-ac1f-4818-9a2a-065a2bc6cab1\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jcw5n" Jul 08 22:15:29.403672 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.403670 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9ba2c75b-169e-4f62-85a4-34171290a5bf-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-kqq85\" (UID: \"9ba2c75b-169e-4f62-85a4-34171290a5bf\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kqq85" Jul 08 22:15:29.403859 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.403708 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ba2c75b-169e-4f62-85a4-34171290a5bf-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-kqq85\" (UID: \"9ba2c75b-169e-4f62-85a4-34171290a5bf\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kqq85" Jul 08 22:15:29.403859 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.403740 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgrdg\" (UniqueName: \"kubernetes.io/projected/9ba2c75b-169e-4f62-85a4-34171290a5bf-kube-api-access-zgrdg\") pod \"kube-state-metrics-7c9dbb8b7d-kqq85\" (UID: \"9ba2c75b-169e-4f62-85a4-34171290a5bf\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kqq85" Jul 08 22:15:29.403859 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.403771 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9ba2c75b-169e-4f62-85a4-34171290a5bf-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-kqq85\" (UID: \"9ba2c75b-169e-4f62-85a4-34171290a5bf\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kqq85" Jul 08 22:15:29.403859 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.403800 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xn6t\" (UniqueName: \"kubernetes.io/projected/5085739c-ac1f-4818-9a2a-065a2bc6cab1-kube-api-access-2xn6t\") pod \"openshift-state-metrics-54cf96dfcf-jcw5n\" (UID: \"5085739c-ac1f-4818-9a2a-065a2bc6cab1\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jcw5n" Jul 08 22:15:29.504787 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.504748 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eb2520ae-ada6-4124-ae5c-8a7bd461d500-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-46rlm\" (UID: \"eb2520ae-ada6-4124-ae5c-8a7bd461d500\") " pod="openshift-monitoring/node-exporter-46rlm" Jul 08 22:15:29.505016 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.504816 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9ba2c75b-169e-4f62-85a4-34171290a5bf-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-kqq85\" (UID: \"9ba2c75b-169e-4f62-85a4-34171290a5bf\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kqq85" Jul 08 22:15:29.505016 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.504842 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9ba2c75b-169e-4f62-85a4-34171290a5bf-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-kqq85\" (UID: \"9ba2c75b-169e-4f62-85a4-34171290a5bf\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kqq85" Jul 08 22:15:29.505016 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.504868 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5085739c-ac1f-4818-9a2a-065a2bc6cab1-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-jcw5n\" (UID: \"5085739c-ac1f-4818-9a2a-065a2bc6cab1\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jcw5n" Jul 08 22:15:29.505016 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.504906 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eb2520ae-ada6-4124-ae5c-8a7bd461d500-sys\") pod \"node-exporter-46rlm\" (UID: \"eb2520ae-ada6-4124-ae5c-8a7bd461d500\") " pod="openshift-monitoring/node-exporter-46rlm" Jul 08 22:15:29.505016 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.504928 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/eb2520ae-ada6-4124-ae5c-8a7bd461d500-node-exporter-wtmp\") pod \"node-exporter-46rlm\" (UID: \"eb2520ae-ada6-4124-ae5c-8a7bd461d500\") " pod="openshift-monitoring/node-exporter-46rlm" Jul 08 22:15:29.505016 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.504944 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbnh6\" (UniqueName: \"kubernetes.io/projected/eb2520ae-ada6-4124-ae5c-8a7bd461d500-kube-api-access-bbnh6\") pod \"node-exporter-46rlm\" (UID: \"eb2520ae-ada6-4124-ae5c-8a7bd461d500\") " pod="openshift-monitoring/node-exporter-46rlm" Jul 08 22:15:29.505334 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.505077 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5085739c-ac1f-4818-9a2a-065a2bc6cab1-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-jcw5n\" (UID: \"5085739c-ac1f-4818-9a2a-065a2bc6cab1\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jcw5n" Jul 08 22:15:29.505334 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:29.505084 2563 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 08 22:15:29.505334 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.505102 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5085739c-ac1f-4818-9a2a-065a2bc6cab1-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-jcw5n\" (UID: \"5085739c-ac1f-4818-9a2a-065a2bc6cab1\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jcw5n" Jul 08 22:15:29.505334 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.505119 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/eb2520ae-ada6-4124-ae5c-8a7bd461d500-node-exporter-tls\") pod \"node-exporter-46rlm\" (UID: \"eb2520ae-ada6-4124-ae5c-8a7bd461d500\") " pod="openshift-monitoring/node-exporter-46rlm" Jul 08 22:15:29.505334 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:29.505161 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ba2c75b-169e-4f62-85a4-34171290a5bf-kube-state-metrics-tls podName:9ba2c75b-169e-4f62-85a4-34171290a5bf nodeName:}" failed. No retries permitted until 2026-07-08 22:15:30.005138817 +0000 UTC m=+61.273158841 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/9ba2c75b-169e-4f62-85a4-34171290a5bf-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-kqq85" (UID: "9ba2c75b-169e-4f62-85a4-34171290a5bf") : secret "kube-state-metrics-tls" not found Jul 08 22:15:29.505334 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:29.505188 2563 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 08 22:15:29.505334 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:29.505245 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5085739c-ac1f-4818-9a2a-065a2bc6cab1-openshift-state-metrics-tls podName:5085739c-ac1f-4818-9a2a-065a2bc6cab1 nodeName:}" failed. No retries permitted until 2026-07-08 22:15:30.005228247 +0000 UTC m=+61.273248267 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/5085739c-ac1f-4818-9a2a-065a2bc6cab1-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-jcw5n" (UID: "5085739c-ac1f-4818-9a2a-065a2bc6cab1") : secret "openshift-state-metrics-tls" not found Jul 08 22:15:29.505334 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.505188 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9ba2c75b-169e-4f62-85a4-34171290a5bf-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-kqq85\" (UID: \"9ba2c75b-169e-4f62-85a4-34171290a5bf\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kqq85" Jul 08 22:15:29.505334 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.505279 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/eb2520ae-ada6-4124-ae5c-8a7bd461d500-node-exporter-accelerators-collector-config\") pod \"node-exporter-46rlm\" (UID: \"eb2520ae-ada6-4124-ae5c-8a7bd461d500\") " pod="openshift-monitoring/node-exporter-46rlm" Jul 08 22:15:29.505334 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.505310 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ba2c75b-169e-4f62-85a4-34171290a5bf-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-kqq85\" (UID: \"9ba2c75b-169e-4f62-85a4-34171290a5bf\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kqq85" Jul 08 22:15:29.505855 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.505356 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/eb2520ae-ada6-4124-ae5c-8a7bd461d500-node-exporter-textfile\") pod \"node-exporter-46rlm\" (UID: \"eb2520ae-ada6-4124-ae5c-8a7bd461d500\") " pod="openshift-monitoring/node-exporter-46rlm" Jul 08 22:15:29.505855 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.505388 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgrdg\" (UniqueName: \"kubernetes.io/projected/9ba2c75b-169e-4f62-85a4-34171290a5bf-kube-api-access-zgrdg\") pod \"kube-state-metrics-7c9dbb8b7d-kqq85\" (UID: \"9ba2c75b-169e-4f62-85a4-34171290a5bf\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kqq85" Jul 08 22:15:29.505855 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.505432 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/eb2520ae-ada6-4124-ae5c-8a7bd461d500-root\") pod \"node-exporter-46rlm\" (UID: \"eb2520ae-ada6-4124-ae5c-8a7bd461d500\") " pod="openshift-monitoring/node-exporter-46rlm" Jul 08 22:15:29.505855 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.505486 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9ba2c75b-169e-4f62-85a4-34171290a5bf-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-kqq85\" (UID: \"9ba2c75b-169e-4f62-85a4-34171290a5bf\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kqq85" Jul 08 22:15:29.505855 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.505566 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb2520ae-ada6-4124-ae5c-8a7bd461d500-metrics-client-ca\") pod \"node-exporter-46rlm\" (UID: \"eb2520ae-ada6-4124-ae5c-8a7bd461d500\") " pod="openshift-monitoring/node-exporter-46rlm" Jul 08 22:15:29.505855 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.505619 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9ba2c75b-169e-4f62-85a4-34171290a5bf-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-kqq85\" (UID: \"9ba2c75b-169e-4f62-85a4-34171290a5bf\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kqq85" Jul 08 22:15:29.505855 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.505649 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xn6t\" (UniqueName: \"kubernetes.io/projected/5085739c-ac1f-4818-9a2a-065a2bc6cab1-kube-api-access-2xn6t\") pod \"openshift-state-metrics-54cf96dfcf-jcw5n\" (UID: \"5085739c-ac1f-4818-9a2a-065a2bc6cab1\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jcw5n" Jul 08 22:15:29.505855 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.505716 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9ba2c75b-169e-4f62-85a4-34171290a5bf-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-kqq85\" (UID: \"9ba2c75b-169e-4f62-85a4-34171290a5bf\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kqq85" Jul 08 22:15:29.505855 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.505725 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5085739c-ac1f-4818-9a2a-065a2bc6cab1-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-jcw5n\" (UID: \"5085739c-ac1f-4818-9a2a-065a2bc6cab1\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jcw5n" Jul 08 22:15:29.506185 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.506143 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ba2c75b-169e-4f62-85a4-34171290a5bf-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-kqq85\" (UID: \"9ba2c75b-169e-4f62-85a4-34171290a5bf\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kqq85" Jul 08 22:15:29.507562 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.507540 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5085739c-ac1f-4818-9a2a-065a2bc6cab1-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-jcw5n\" (UID: \"5085739c-ac1f-4818-9a2a-065a2bc6cab1\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jcw5n" Jul 08 22:15:29.507952 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.507936 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9ba2c75b-169e-4f62-85a4-34171290a5bf-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-kqq85\" (UID: \"9ba2c75b-169e-4f62-85a4-34171290a5bf\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kqq85" Jul 08 22:15:29.514603 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.514575 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgrdg\" (UniqueName: \"kubernetes.io/projected/9ba2c75b-169e-4f62-85a4-34171290a5bf-kube-api-access-zgrdg\") pod \"kube-state-metrics-7c9dbb8b7d-kqq85\" (UID: \"9ba2c75b-169e-4f62-85a4-34171290a5bf\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kqq85" Jul 08 22:15:29.514802 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.514778 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xn6t\" (UniqueName: \"kubernetes.io/projected/5085739c-ac1f-4818-9a2a-065a2bc6cab1-kube-api-access-2xn6t\") pod \"openshift-state-metrics-54cf96dfcf-jcw5n\" (UID: \"5085739c-ac1f-4818-9a2a-065a2bc6cab1\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jcw5n" Jul 08 22:15:29.607013 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.606898 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eb2520ae-ada6-4124-ae5c-8a7bd461d500-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-46rlm\" (UID: \"eb2520ae-ada6-4124-ae5c-8a7bd461d500\") " pod="openshift-monitoring/node-exporter-46rlm" Jul 08 22:15:29.607013 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.606992 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eb2520ae-ada6-4124-ae5c-8a7bd461d500-sys\") pod \"node-exporter-46rlm\" (UID: \"eb2520ae-ada6-4124-ae5c-8a7bd461d500\") " pod="openshift-monitoring/node-exporter-46rlm" Jul 08 22:15:29.607013 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.607013 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/eb2520ae-ada6-4124-ae5c-8a7bd461d500-node-exporter-wtmp\") pod \"node-exporter-46rlm\" (UID: \"eb2520ae-ada6-4124-ae5c-8a7bd461d500\") " pod="openshift-monitoring/node-exporter-46rlm" Jul 08 22:15:29.607297 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.607031 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbnh6\" (UniqueName: \"kubernetes.io/projected/eb2520ae-ada6-4124-ae5c-8a7bd461d500-kube-api-access-bbnh6\") pod \"node-exporter-46rlm\" (UID: \"eb2520ae-ada6-4124-ae5c-8a7bd461d500\") " pod="openshift-monitoring/node-exporter-46rlm" Jul 08 22:15:29.607297 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.607075 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/eb2520ae-ada6-4124-ae5c-8a7bd461d500-node-exporter-tls\") pod \"node-exporter-46rlm\" (UID: \"eb2520ae-ada6-4124-ae5c-8a7bd461d500\") " pod="openshift-monitoring/node-exporter-46rlm" Jul 08 22:15:29.607297 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.607101 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eb2520ae-ada6-4124-ae5c-8a7bd461d500-sys\") pod \"node-exporter-46rlm\" (UID: \"eb2520ae-ada6-4124-ae5c-8a7bd461d500\") " pod="openshift-monitoring/node-exporter-46rlm" Jul 08 22:15:29.607297 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.607107 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/eb2520ae-ada6-4124-ae5c-8a7bd461d500-node-exporter-accelerators-collector-config\") pod \"node-exporter-46rlm\" (UID: \"eb2520ae-ada6-4124-ae5c-8a7bd461d500\") " pod="openshift-monitoring/node-exporter-46rlm" Jul 08 22:15:29.607297 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.607173 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/eb2520ae-ada6-4124-ae5c-8a7bd461d500-node-exporter-textfile\") pod \"node-exporter-46rlm\" (UID: \"eb2520ae-ada6-4124-ae5c-8a7bd461d500\") " pod="openshift-monitoring/node-exporter-46rlm" Jul 08 22:15:29.607297 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.607186 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/eb2520ae-ada6-4124-ae5c-8a7bd461d500-node-exporter-wtmp\") pod \"node-exporter-46rlm\" (UID: \"eb2520ae-ada6-4124-ae5c-8a7bd461d500\") " pod="openshift-monitoring/node-exporter-46rlm" Jul 08 22:15:29.607297 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.607206 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/eb2520ae-ada6-4124-ae5c-8a7bd461d500-root\") pod \"node-exporter-46rlm\" (UID: \"eb2520ae-ada6-4124-ae5c-8a7bd461d500\") " pod="openshift-monitoring/node-exporter-46rlm" Jul 08 22:15:29.607297 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.607257 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb2520ae-ada6-4124-ae5c-8a7bd461d500-metrics-client-ca\") pod \"node-exporter-46rlm\" (UID: \"eb2520ae-ada6-4124-ae5c-8a7bd461d500\") " pod="openshift-monitoring/node-exporter-46rlm" Jul 08 22:15:29.607297 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:29.607267 2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 08 22:15:29.607673 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:29.607324 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb2520ae-ada6-4124-ae5c-8a7bd461d500-node-exporter-tls podName:eb2520ae-ada6-4124-ae5c-8a7bd461d500 nodeName:}" failed. No retries permitted until 2026-07-08 22:15:30.107305185 +0000 UTC m=+61.375325207 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/eb2520ae-ada6-4124-ae5c-8a7bd461d500-node-exporter-tls") pod "node-exporter-46rlm" (UID: "eb2520ae-ada6-4124-ae5c-8a7bd461d500") : secret "node-exporter-tls" not found Jul 08 22:15:29.607673 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.607327 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/eb2520ae-ada6-4124-ae5c-8a7bd461d500-root\") pod \"node-exporter-46rlm\" (UID: \"eb2520ae-ada6-4124-ae5c-8a7bd461d500\") " pod="openshift-monitoring/node-exporter-46rlm" Jul 08 22:15:29.607673 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.607481 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/eb2520ae-ada6-4124-ae5c-8a7bd461d500-node-exporter-textfile\") pod \"node-exporter-46rlm\" (UID: \"eb2520ae-ada6-4124-ae5c-8a7bd461d500\") " pod="openshift-monitoring/node-exporter-46rlm" Jul 08 22:15:29.607673 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.607667 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/eb2520ae-ada6-4124-ae5c-8a7bd461d500-node-exporter-accelerators-collector-config\") pod \"node-exporter-46rlm\" (UID: \"eb2520ae-ada6-4124-ae5c-8a7bd461d500\") " pod="openshift-monitoring/node-exporter-46rlm" Jul 08 22:15:29.607796 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.607751 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb2520ae-ada6-4124-ae5c-8a7bd461d500-metrics-client-ca\") pod \"node-exporter-46rlm\" (UID: \"eb2520ae-ada6-4124-ae5c-8a7bd461d500\") " pod="openshift-monitoring/node-exporter-46rlm" Jul 08 22:15:29.609323 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.609301 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eb2520ae-ada6-4124-ae5c-8a7bd461d500-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-46rlm\" (UID: \"eb2520ae-ada6-4124-ae5c-8a7bd461d500\") " pod="openshift-monitoring/node-exporter-46rlm" Jul 08 22:15:29.616550 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:29.616517 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbnh6\" (UniqueName: \"kubernetes.io/projected/eb2520ae-ada6-4124-ae5c-8a7bd461d500-kube-api-access-bbnh6\") pod \"node-exporter-46rlm\" (UID: \"eb2520ae-ada6-4124-ae5c-8a7bd461d500\") " pod="openshift-monitoring/node-exporter-46rlm" Jul 08 22:15:30.011150 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:30.011111 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9ba2c75b-169e-4f62-85a4-34171290a5bf-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-kqq85\" (UID: \"9ba2c75b-169e-4f62-85a4-34171290a5bf\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kqq85" Jul 08 22:15:30.011150 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:30.011160 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5085739c-ac1f-4818-9a2a-065a2bc6cab1-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-jcw5n\" (UID: \"5085739c-ac1f-4818-9a2a-065a2bc6cab1\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jcw5n" Jul 08 22:15:30.011683 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:30.011281 2563 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 08 22:15:30.011683 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:30.011366 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ba2c75b-169e-4f62-85a4-34171290a5bf-kube-state-metrics-tls podName:9ba2c75b-169e-4f62-85a4-34171290a5bf nodeName:}" failed. No retries permitted until 2026-07-08 22:15:31.011344348 +0000 UTC m=+62.279364371 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/9ba2c75b-169e-4f62-85a4-34171290a5bf-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-kqq85" (UID: "9ba2c75b-169e-4f62-85a4-34171290a5bf") : secret "kube-state-metrics-tls" not found Jul 08 22:15:30.013640 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:30.013619 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5085739c-ac1f-4818-9a2a-065a2bc6cab1-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-jcw5n\" (UID: \"5085739c-ac1f-4818-9a2a-065a2bc6cab1\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jcw5n" Jul 08 22:15:30.112529 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:30.112497 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/eb2520ae-ada6-4124-ae5c-8a7bd461d500-node-exporter-tls\") pod \"node-exporter-46rlm\" (UID: \"eb2520ae-ada6-4124-ae5c-8a7bd461d500\") " pod="openshift-monitoring/node-exporter-46rlm" Jul 08 22:15:30.114786 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:30.114760 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/eb2520ae-ada6-4124-ae5c-8a7bd461d500-node-exporter-tls\") pod \"node-exporter-46rlm\" (UID: \"eb2520ae-ada6-4124-ae5c-8a7bd461d500\") " pod="openshift-monitoring/node-exporter-46rlm" Jul 08 22:15:30.255702 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:30.255657 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jcw5n" Jul 08 22:15:30.301068 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:30.301029 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-46rlm" Jul 08 22:15:30.394599 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:30.394575 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-jcw5n"] Jul 08 22:15:30.397254 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:15:30.397222 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5085739c_ac1f_4818_9a2a_065a2bc6cab1.slice/crio-000ba8657e7fba7eef68f5388bebbf804690448eeae38dd4d2426f0e0c9cc7bf WatchSource:0}: Error finding container 000ba8657e7fba7eef68f5388bebbf804690448eeae38dd4d2426f0e0c9cc7bf: Status 404 returned error can't find the container with id 000ba8657e7fba7eef68f5388bebbf804690448eeae38dd4d2426f0e0c9cc7bf Jul 08 22:15:30.704377 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:30.704269 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jcw5n" event={"ID":"5085739c-ac1f-4818-9a2a-065a2bc6cab1","Type":"ContainerStarted","Data":"744f4ed991b1e922b424bbb17b1090a348ae13d92d0ba76ecef43087b0c0f729"} Jul 08 22:15:30.704377 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:30.704331 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jcw5n" event={"ID":"5085739c-ac1f-4818-9a2a-065a2bc6cab1","Type":"ContainerStarted","Data":"e8d679a4c16ce4d6dc3db9799d8dc918da702588d1f0067f2e6d079ab39239d0"} Jul 08 22:15:30.704377 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:30.704352 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jcw5n" event={"ID":"5085739c-ac1f-4818-9a2a-065a2bc6cab1","Type":"ContainerStarted","Data":"000ba8657e7fba7eef68f5388bebbf804690448eeae38dd4d2426f0e0c9cc7bf"} Jul 08 22:15:30.705404 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:30.705377 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-46rlm" event={"ID":"eb2520ae-ada6-4124-ae5c-8a7bd461d500","Type":"ContainerStarted","Data":"1c50afb6da7f552dc527330e583c02e91cf9198731d097e4cdbf6130f034afa8"} Jul 08 22:15:31.020109 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:31.020075 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9ba2c75b-169e-4f62-85a4-34171290a5bf-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-kqq85\" (UID: \"9ba2c75b-169e-4f62-85a4-34171290a5bf\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kqq85" Jul 08 22:15:31.022467 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:31.022438 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9ba2c75b-169e-4f62-85a4-34171290a5bf-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-kqq85\" (UID: \"9ba2c75b-169e-4f62-85a4-34171290a5bf\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kqq85" Jul 08 22:15:31.174302 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:31.174266 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kqq85" Jul 08 22:15:31.322347 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:31.322033 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kqq85"] Jul 08 22:15:31.529582 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:15:31.529553 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ba2c75b_169e_4f62_85a4_34171290a5bf.slice/crio-ac003f01fddb46936a812d2062d3905c8cfddc0780b3d824f328fd71af1e6833 WatchSource:0}: Error finding container ac003f01fddb46936a812d2062d3905c8cfddc0780b3d824f328fd71af1e6833: Status 404 returned error can't find the container with id ac003f01fddb46936a812d2062d3905c8cfddc0780b3d824f328fd71af1e6833 Jul 08 22:15:31.715844 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:31.715799 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kqq85" event={"ID":"9ba2c75b-169e-4f62-85a4-34171290a5bf","Type":"ContainerStarted","Data":"ac003f01fddb46936a812d2062d3905c8cfddc0780b3d824f328fd71af1e6833"} Jul 08 22:15:31.718166 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:31.718136 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jcw5n" event={"ID":"5085739c-ac1f-4818-9a2a-065a2bc6cab1","Type":"ContainerStarted","Data":"e27813e311255c584ecbe0e2b2499cdcb58acd4fc226b8168987def3fc7a8b2e"} Jul 08 22:15:32.604775 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:32.604740 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bltvn" Jul 08 22:15:32.634366 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:32.634299 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jcw5n" podStartSLOduration=2.589212624 podStartE2EDuration="3.634276579s" totalImagesPullingTime="1.045063955s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 22:15:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:15:31.737840348 +0000 UTC m=+63.005860377" watchObservedRunningTime="2026-07-08 22:15:32.634276579 +0000 UTC m=+63.902296606" Jul 08 22:15:32.723275 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:32.723238 2563 generic.go:356] "Generic (PLEG): container finished" podID="eb2520ae-ada6-4124-ae5c-8a7bd461d500" containerID="1085377798e6bbddd28c7435e8c66e46ec2d768ef09ed99b2f83ba6474496042" exitCode=0 Jul 08 22:15:32.723461 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:32.723329 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-46rlm" event={"ID":"eb2520ae-ada6-4124-ae5c-8a7bd461d500","Type":"ContainerDied","Data":"1085377798e6bbddd28c7435e8c66e46ec2d768ef09ed99b2f83ba6474496042"} Jul 08 22:15:33.729344 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:33.729310 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kqq85" event={"ID":"9ba2c75b-169e-4f62-85a4-34171290a5bf","Type":"ContainerStarted","Data":"e11ab3105cf6fb595c4603232eef3888e8289ceaa7e469e2848fd30e947c6a0a"} Jul 08 22:15:33.729797 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:33.729382 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kqq85" event={"ID":"9ba2c75b-169e-4f62-85a4-34171290a5bf","Type":"ContainerStarted","Data":"4ae16082cd458d1f4daeff3ed625ec174bf849fe2108d79f3bf58369cfc2652d"} Jul 08 22:15:33.729797 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:33.729405 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kqq85" event={"ID":"9ba2c75b-169e-4f62-85a4-34171290a5bf","Type":"ContainerStarted","Data":"71dc17b64e7f34872022404f662fd1fe85723dcfc0dc99f6231eb7885a15f014"} Jul 08 22:15:33.731408 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:33.731383 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-46rlm" event={"ID":"eb2520ae-ada6-4124-ae5c-8a7bd461d500","Type":"ContainerStarted","Data":"d0a3571961d94c0eeb31ede62b1046ed1e1fe991442cb3121e6c006e88ea6d29"} Jul 08 22:15:33.731531 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:33.731430 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-46rlm" event={"ID":"eb2520ae-ada6-4124-ae5c-8a7bd461d500","Type":"ContainerStarted","Data":"a225248c965597531413b09d022b02dff8cb2aecb08e201082908679c200f096"} Jul 08 22:15:33.751229 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:33.751171 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kqq85" podStartSLOduration=3.45921045 podStartE2EDuration="4.7511479s" totalImagesPullingTime="1.29193745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 22:15:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:15:33.750067991 +0000 UTC m=+65.018088019" watchObservedRunningTime="2026-07-08 22:15:33.7511479 +0000 UTC m=+65.019167928" Jul 08 22:15:33.774238 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:33.774184 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-46rlm" podStartSLOduration=3.0792712 podStartE2EDuration="4.774168485s" totalImagesPullingTime="1.610830056s" totalInitContainerRuntime="84.067229ms" isStatefulPod=true podCreationTimestamp="2026-07-08 22:15:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:15:33.773244613 +0000 UTC m=+65.041264642" watchObservedRunningTime="2026-07-08 22:15:33.774168485 +0000 UTC m=+65.042188513" Jul 08 22:15:34.149678 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.149641 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-thl2v"] Jul 08 22:15:34.151953 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.151928 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-thl2v" Jul 08 22:15:34.154491 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.154467 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-wh5xd\"" Jul 08 22:15:34.154625 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.154503 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 08 22:15:34.158721 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.158687 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-658dc76d8d-lx2hh"] Jul 08 22:15:34.166546 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.166517 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-thl2v"] Jul 08 22:15:34.194672 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.194640 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c4585cb94-tw6td"] Jul 08 22:15:34.196804 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.196787 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c4585cb94-tw6td" Jul 08 22:15:34.207721 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.207697 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c4585cb94-tw6td"] Jul 08 22:15:34.250773 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.250733 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6knq\" (UniqueName: \"kubernetes.io/projected/6c022be9-ee23-4a42-b174-f502add283f7-kube-api-access-p6knq\") pod \"console-7c4585cb94-tw6td\" (UID: \"6c022be9-ee23-4a42-b174-f502add283f7\") " pod="openshift-console/console-7c4585cb94-tw6td" Jul 08 22:15:34.250773 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.250773 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdpm2\" (UniqueName: \"kubernetes.io/projected/f38983c4-b714-4da9-9336-28b1da2c78b5-kube-api-access-mdpm2\") pod \"monitoring-plugin-d5dcff7cf-thl2v\" (UID: \"f38983c4-b714-4da9-9336-28b1da2c78b5\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-thl2v" Jul 08 22:15:34.250976 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.250791 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6c022be9-ee23-4a42-b174-f502add283f7-service-ca\") pod \"console-7c4585cb94-tw6td\" (UID: \"6c022be9-ee23-4a42-b174-f502add283f7\") " pod="openshift-console/console-7c4585cb94-tw6td" Jul 08 22:15:34.250976 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.250869 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6c022be9-ee23-4a42-b174-f502add283f7-console-serving-cert\") pod \"console-7c4585cb94-tw6td\" (UID: \"6c022be9-ee23-4a42-b174-f502add283f7\") " pod="openshift-console/console-7c4585cb94-tw6td" Jul 08 22:15:34.250976 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.250956 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c022be9-ee23-4a42-b174-f502add283f7-trusted-ca-bundle\") pod \"console-7c4585cb94-tw6td\" (UID: \"6c022be9-ee23-4a42-b174-f502add283f7\") " pod="openshift-console/console-7c4585cb94-tw6td" Jul 08 22:15:34.251125 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.250984 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f38983c4-b714-4da9-9336-28b1da2c78b5-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-thl2v\" (UID: \"f38983c4-b714-4da9-9336-28b1da2c78b5\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-thl2v" Jul 08 22:15:34.251125 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.251004 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6c022be9-ee23-4a42-b174-f502add283f7-console-config\") pod \"console-7c4585cb94-tw6td\" (UID: \"6c022be9-ee23-4a42-b174-f502add283f7\") " pod="openshift-console/console-7c4585cb94-tw6td" Jul 08 22:15:34.251125 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.251027 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6c022be9-ee23-4a42-b174-f502add283f7-oauth-serving-cert\") pod \"console-7c4585cb94-tw6td\" (UID: \"6c022be9-ee23-4a42-b174-f502add283f7\") " pod="openshift-console/console-7c4585cb94-tw6td" Jul 08 22:15:34.251125 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.251096 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6c022be9-ee23-4a42-b174-f502add283f7-console-oauth-config\") pod \"console-7c4585cb94-tw6td\" (UID: \"6c022be9-ee23-4a42-b174-f502add283f7\") " pod="openshift-console/console-7c4585cb94-tw6td" Jul 08 22:15:34.352482 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.352380 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6knq\" (UniqueName: \"kubernetes.io/projected/6c022be9-ee23-4a42-b174-f502add283f7-kube-api-access-p6knq\") pod \"console-7c4585cb94-tw6td\" (UID: \"6c022be9-ee23-4a42-b174-f502add283f7\") " pod="openshift-console/console-7c4585cb94-tw6td" Jul 08 22:15:34.352482 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.352423 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdpm2\" (UniqueName: \"kubernetes.io/projected/f38983c4-b714-4da9-9336-28b1da2c78b5-kube-api-access-mdpm2\") pod \"monitoring-plugin-d5dcff7cf-thl2v\" (UID: \"f38983c4-b714-4da9-9336-28b1da2c78b5\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-thl2v" Jul 08 22:15:34.352482 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.352444 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6c022be9-ee23-4a42-b174-f502add283f7-service-ca\") pod \"console-7c4585cb94-tw6td\" (UID: \"6c022be9-ee23-4a42-b174-f502add283f7\") " pod="openshift-console/console-7c4585cb94-tw6td" Jul 08 22:15:34.352482 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.352470 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6c022be9-ee23-4a42-b174-f502add283f7-console-serving-cert\") pod \"console-7c4585cb94-tw6td\" (UID: \"6c022be9-ee23-4a42-b174-f502add283f7\") " pod="openshift-console/console-7c4585cb94-tw6td" Jul 08 22:15:34.352808 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.352698 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c022be9-ee23-4a42-b174-f502add283f7-trusted-ca-bundle\") pod \"console-7c4585cb94-tw6td\" (UID: \"6c022be9-ee23-4a42-b174-f502add283f7\") " pod="openshift-console/console-7c4585cb94-tw6td" Jul 08 22:15:34.352808 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.352740 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f38983c4-b714-4da9-9336-28b1da2c78b5-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-thl2v\" (UID: \"f38983c4-b714-4da9-9336-28b1da2c78b5\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-thl2v" Jul 08 22:15:34.352808 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.352765 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6c022be9-ee23-4a42-b174-f502add283f7-console-config\") pod \"console-7c4585cb94-tw6td\" (UID: \"6c022be9-ee23-4a42-b174-f502add283f7\") " pod="openshift-console/console-7c4585cb94-tw6td" Jul 08 22:15:34.352808 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.352806 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6c022be9-ee23-4a42-b174-f502add283f7-oauth-serving-cert\") pod \"console-7c4585cb94-tw6td\" (UID: \"6c022be9-ee23-4a42-b174-f502add283f7\") " pod="openshift-console/console-7c4585cb94-tw6td" Jul 08 22:15:34.353039 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.352834 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6c022be9-ee23-4a42-b174-f502add283f7-console-oauth-config\") pod \"console-7c4585cb94-tw6td\" (UID: \"6c022be9-ee23-4a42-b174-f502add283f7\") " pod="openshift-console/console-7c4585cb94-tw6td" Jul 08 22:15:34.353039 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:34.352853 2563 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 08 22:15:34.353147 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:15:34.353049 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f38983c4-b714-4da9-9336-28b1da2c78b5-monitoring-plugin-cert podName:f38983c4-b714-4da9-9336-28b1da2c78b5 nodeName:}" failed. No retries permitted until 2026-07-08 22:15:34.85302701 +0000 UTC m=+66.121047029 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/f38983c4-b714-4da9-9336-28b1da2c78b5-monitoring-plugin-cert") pod "monitoring-plugin-d5dcff7cf-thl2v" (UID: "f38983c4-b714-4da9-9336-28b1da2c78b5") : secret "monitoring-plugin-cert" not found Jul 08 22:15:34.353392 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.353362 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6c022be9-ee23-4a42-b174-f502add283f7-service-ca\") pod \"console-7c4585cb94-tw6td\" (UID: \"6c022be9-ee23-4a42-b174-f502add283f7\") " pod="openshift-console/console-7c4585cb94-tw6td" Jul 08 22:15:34.353524 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.353513 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c022be9-ee23-4a42-b174-f502add283f7-trusted-ca-bundle\") pod \"console-7c4585cb94-tw6td\" (UID: \"6c022be9-ee23-4a42-b174-f502add283f7\") " pod="openshift-console/console-7c4585cb94-tw6td" Jul 08 22:15:34.353600 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.353575 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6c022be9-ee23-4a42-b174-f502add283f7-oauth-serving-cert\") pod \"console-7c4585cb94-tw6td\" (UID: \"6c022be9-ee23-4a42-b174-f502add283f7\") " pod="openshift-console/console-7c4585cb94-tw6td" Jul 08 22:15:34.353600 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.353582 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6c022be9-ee23-4a42-b174-f502add283f7-console-config\") pod \"console-7c4585cb94-tw6td\" (UID: \"6c022be9-ee23-4a42-b174-f502add283f7\") " pod="openshift-console/console-7c4585cb94-tw6td" Jul 08 22:15:34.355056 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.355033 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6c022be9-ee23-4a42-b174-f502add283f7-console-serving-cert\") pod \"console-7c4585cb94-tw6td\" (UID: \"6c022be9-ee23-4a42-b174-f502add283f7\") " pod="openshift-console/console-7c4585cb94-tw6td" Jul 08 22:15:34.355286 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.355266 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6c022be9-ee23-4a42-b174-f502add283f7-console-oauth-config\") pod \"console-7c4585cb94-tw6td\" (UID: \"6c022be9-ee23-4a42-b174-f502add283f7\") " pod="openshift-console/console-7c4585cb94-tw6td" Jul 08 22:15:34.363387 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.363358 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdpm2\" (UniqueName: \"kubernetes.io/projected/f38983c4-b714-4da9-9336-28b1da2c78b5-kube-api-access-mdpm2\") pod \"monitoring-plugin-d5dcff7cf-thl2v\" (UID: \"f38983c4-b714-4da9-9336-28b1da2c78b5\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-thl2v" Jul 08 22:15:34.364213 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.364194 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6knq\" (UniqueName: \"kubernetes.io/projected/6c022be9-ee23-4a42-b174-f502add283f7-kube-api-access-p6knq\") pod \"console-7c4585cb94-tw6td\" (UID: \"6c022be9-ee23-4a42-b174-f502add283f7\") " pod="openshift-console/console-7c4585cb94-tw6td" Jul 08 22:15:34.505416 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.505373 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c4585cb94-tw6td" Jul 08 22:15:34.646323 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.646244 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c4585cb94-tw6td"] Jul 08 22:15:34.649364 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:15:34.649337 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c022be9_ee23_4a42_b174_f502add283f7.slice/crio-0087bbc1d942f50f5721ec6ecbb91f4912df245f135de9f6b365302ad95ea091 WatchSource:0}: Error finding container 0087bbc1d942f50f5721ec6ecbb91f4912df245f135de9f6b365302ad95ea091: Status 404 returned error can't find the container with id 0087bbc1d942f50f5721ec6ecbb91f4912df245f135de9f6b365302ad95ea091 Jul 08 22:15:34.720509 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.720469 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c5f6bd69f-2l4z7" Jul 08 22:15:34.720670 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.720549 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c5f6bd69f-2l4z7" Jul 08 22:15:34.722102 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.722072 2563 patch_prober.go:28] interesting pod/console-5c5f6bd69f-2l4z7 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.11:8443/health\": dial tcp 10.134.0.11:8443: connect: connection refused" start-of-body= Jul 08 22:15:34.722244 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.722126 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-5c5f6bd69f-2l4z7" podUID="5d62c766-320a-4fea-ac5a-23db8702e521" containerName="console" probeResult="failure" output="Get \"https://10.134.0.11:8443/health\": dial tcp 10.134.0.11:8443: connect: connection refused" Jul 08 22:15:34.736902 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.736842 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c4585cb94-tw6td" event={"ID":"6c022be9-ee23-4a42-b174-f502add283f7","Type":"ContainerStarted","Data":"007e9c5d458a47d85cb82352fca69e084c53da2b21aa1dc9151de85f0f5e5577"} Jul 08 22:15:34.737340 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.736926 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c4585cb94-tw6td" event={"ID":"6c022be9-ee23-4a42-b174-f502add283f7","Type":"ContainerStarted","Data":"0087bbc1d942f50f5721ec6ecbb91f4912df245f135de9f6b365302ad95ea091"} Jul 08 22:15:34.755589 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.755535 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c4585cb94-tw6td" podStartSLOduration=0.75551656 podStartE2EDuration="755.51656ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 22:15:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:15:34.754666908 +0000 UTC m=+66.022686949" watchObservedRunningTime="2026-07-08 22:15:34.75551656 +0000 UTC m=+66.023536587" Jul 08 22:15:34.858032 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.857658 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f38983c4-b714-4da9-9336-28b1da2c78b5-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-thl2v\" (UID: \"f38983c4-b714-4da9-9336-28b1da2c78b5\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-thl2v" Jul 08 22:15:34.860026 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.860006 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f38983c4-b714-4da9-9336-28b1da2c78b5-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-thl2v\" (UID: \"f38983c4-b714-4da9-9336-28b1da2c78b5\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-thl2v" Jul 08 22:15:34.958307 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.958219 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a354dc2-56c0-4280-96d1-b560d70be2b2-original-pull-secret\") pod \"global-pull-secret-syncer-fffds\" (UID: \"4a354dc2-56c0-4280-96d1-b560d70be2b2\") " pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:15:34.961240 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.961216 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 22:15:34.971638 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:34.971601 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a354dc2-56c0-4280-96d1-b560d70be2b2-original-pull-secret\") pod \"global-pull-secret-syncer-fffds\" (UID: \"4a354dc2-56c0-4280-96d1-b560d70be2b2\") " pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:15:35.059095 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.059054 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c5c6b58-e07d-4d72-af33-ebe169402d4e-metrics-certs\") pod \"network-metrics-daemon-jhpj6\" (UID: \"7c5c6b58-e07d-4d72-af33-ebe169402d4e\") " pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:15:35.061919 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.061878 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 22:15:35.062990 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.062960 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-thl2v" Jul 08 22:15:35.071703 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.071676 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c5c6b58-e07d-4d72-af33-ebe169402d4e-metrics-certs\") pod \"network-metrics-daemon-jhpj6\" (UID: \"7c5c6b58-e07d-4d72-af33-ebe169402d4e\") " pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:15:35.135810 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.135786 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bwnr2\"" Jul 08 22:15:35.144135 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.144112 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jhpj6" Jul 08 22:15:35.146402 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.146375 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fffds" Jul 08 22:15:35.159997 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.159963 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnbkg\" (UniqueName: \"kubernetes.io/projected/6a151365-c622-4361-862f-ed202f3045f7-kube-api-access-mnbkg\") pod \"network-check-target-98x2n\" (UID: \"6a151365-c622-4361-862f-ed202f3045f7\") " pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:15:35.163015 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.162991 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 22:15:35.172821 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.172792 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 22:15:35.184331 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.184299 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnbkg\" (UniqueName: \"kubernetes.io/projected/6a151365-c622-4361-862f-ed202f3045f7-kube-api-access-mnbkg\") pod \"network-check-target-98x2n\" (UID: \"6a151365-c622-4361-862f-ed202f3045f7\") " pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:15:35.194521 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.194486 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-thl2v"] Jul 08 22:15:35.201914 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:15:35.200279 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf38983c4_b714_4da9_9336_28b1da2c78b5.slice/crio-12489b2a5a4bd9c32bb86fb1ff314fd171baad73db09d36a71456d8ea842d642 WatchSource:0}: Error finding container 12489b2a5a4bd9c32bb86fb1ff314fd171baad73db09d36a71456d8ea842d642: Status 404 returned error can't find the container with id 12489b2a5a4bd9c32bb86fb1ff314fd171baad73db09d36a71456d8ea842d642 Jul 08 22:15:35.299043 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.299009 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jhpj6"] Jul 08 22:15:35.301957 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:15:35.301930 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c5c6b58_e07d_4d72_af33_ebe169402d4e.slice/crio-9b5f05164db413040f567405711668526ba1d287b92af0ea7efff6cfaa00d3ba WatchSource:0}: Error finding container 9b5f05164db413040f567405711668526ba1d287b92af0ea7efff6cfaa00d3ba: Status 404 returned error can't find the container with id 9b5f05164db413040f567405711668526ba1d287b92af0ea7efff6cfaa00d3ba Jul 08 22:15:35.309821 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.309796 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fffds"] Jul 08 22:15:35.312393 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:15:35.312366 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a354dc2_56c0_4280_96d1_b560d70be2b2.slice/crio-68e8fcea6bacdc6214f8badf800877ad1162750ce98c236830de345e3a13005c WatchSource:0}: Error finding container 68e8fcea6bacdc6214f8badf800877ad1162750ce98c236830de345e3a13005c: Status 404 returned error can't find the container with id 68e8fcea6bacdc6214f8badf800877ad1162750ce98c236830de345e3a13005c Jul 08 22:15:35.457796 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.457754 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-4rrdh\"" Jul 08 22:15:35.465424 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.465337 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:15:35.566363 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.566326 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 22:15:35.575967 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.575716 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.579043 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.579013 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 08 22:15:35.579524 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.579502 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-gb5wt\"" Jul 08 22:15:35.580140 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.580096 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 08 22:15:35.580357 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.580325 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 08 22:15:35.580437 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.580386 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 08 22:15:35.580547 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.580217 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 08 22:15:35.581153 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.580626 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4t4b56c4nce9k\"" Jul 08 22:15:35.581153 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.580271 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 08 22:15:35.581153 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.580742 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 08 22:15:35.581153 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.581009 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 08 22:15:35.581372 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.580163 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 08 22:15:35.581692 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.580117 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 08 22:15:35.581850 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.581830 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 08 22:15:35.582118 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.582005 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 08 22:15:35.584555 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.584525 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 08 22:15:35.585246 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.584612 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 08 22:15:35.585246 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.585067 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 08 22:15:35.589083 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.588869 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 22:15:35.641410 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.641374 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-98x2n"] Jul 08 22:15:35.645438 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:15:35.645403 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a151365_c622_4361_862f_ed202f3045f7.slice/crio-c6ac8564410d220ca5f39e4287e0f411f2bdd464424694eb6558f161ca9acce9 WatchSource:0}: Error finding container c6ac8564410d220ca5f39e4287e0f411f2bdd464424694eb6558f161ca9acce9: Status 404 returned error can't find the container with id c6ac8564410d220ca5f39e4287e0f411f2bdd464424694eb6558f161ca9acce9 Jul 08 22:15:35.666776 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.666744 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.667011 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.666796 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-web-config\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.667011 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.666853 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.667011 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.666915 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.667011 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.666934 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.667011 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.666952 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.667011 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.666973 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.667336 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.667040 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.667336 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.667074 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-config\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.667336 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.667096 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.667336 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.667117 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.667336 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.667152 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/35556def-a4de-4ccf-984f-7700074c610c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.667336 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.667167 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6njg\" (UniqueName: \"kubernetes.io/projected/35556def-a4de-4ccf-984f-7700074c610c-kube-api-access-m6njg\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.667336 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.667189 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.667336 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.667215 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.667336 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.667229 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/35556def-a4de-4ccf-984f-7700074c610c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.667336 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.667253 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/35556def-a4de-4ccf-984f-7700074c610c-config-out\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.667336 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.667273 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.667336 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.667319 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.668018 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.667352 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.687123 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.687093 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vrwxr" Jul 08 22:15:35.743144 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.743103 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-98x2n" event={"ID":"6a151365-c622-4361-862f-ed202f3045f7","Type":"ContainerStarted","Data":"c6ac8564410d220ca5f39e4287e0f411f2bdd464424694eb6558f161ca9acce9"} Jul 08 22:15:35.744452 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.744426 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fffds" event={"ID":"4a354dc2-56c0-4280-96d1-b560d70be2b2","Type":"ContainerStarted","Data":"68e8fcea6bacdc6214f8badf800877ad1162750ce98c236830de345e3a13005c"} Jul 08 22:15:35.745588 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.745569 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-thl2v" event={"ID":"f38983c4-b714-4da9-9336-28b1da2c78b5","Type":"ContainerStarted","Data":"12489b2a5a4bd9c32bb86fb1ff314fd171baad73db09d36a71456d8ea842d642"} Jul 08 22:15:35.746758 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.746737 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jhpj6" event={"ID":"7c5c6b58-e07d-4d72-af33-ebe169402d4e","Type":"ContainerStarted","Data":"9b5f05164db413040f567405711668526ba1d287b92af0ea7efff6cfaa00d3ba"} Jul 08 22:15:35.768019 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.767984 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.768184 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.768042 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.768253 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.768193 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/35556def-a4de-4ccf-984f-7700074c610c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.768305 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.768256 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/35556def-a4de-4ccf-984f-7700074c610c-config-out\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.768305 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.768278 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.768391 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.768306 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.768391 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.768342 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.768492 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.768395 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.768492 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.768436 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-web-config\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.768492 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.768469 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.768649 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.768505 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.768649 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.768530 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.768649 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.768571 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.768649 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.768619 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.768839 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.768683 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.768839 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.768731 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-config\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.768839 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.768772 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.768839 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.768763 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.768839 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.768825 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.769120 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.769028 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.769120 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.769111 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/35556def-a4de-4ccf-984f-7700074c610c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.769221 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.769141 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6njg\" (UniqueName: \"kubernetes.io/projected/35556def-a4de-4ccf-984f-7700074c610c-kube-api-access-m6njg\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.771078 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.770343 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.771078 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.770474 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.771078 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.770995 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.772263 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.771806 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.772263 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.772101 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.772263 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.768725 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/35556def-a4de-4ccf-984f-7700074c610c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.772554 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.772273 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/35556def-a4de-4ccf-984f-7700074c610c-config-out\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.772554 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.772426 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.772657 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.772572 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.773073 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.773048 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.773576 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.773550 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.774458 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.774425 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.774714 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.774692 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/35556def-a4de-4ccf-984f-7700074c610c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.774954 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.774931 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.775130 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.775114 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-config\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.775221 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.775201 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.775503 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.775488 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-web-config\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.777203 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.777183 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6njg\" (UniqueName: \"kubernetes.io/projected/35556def-a4de-4ccf-984f-7700074c610c-kube-api-access-m6njg\") pod \"prometheus-k8s-0\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:35.893386 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:35.893341 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:36.101659 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:36.101616 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 22:15:36.106997 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:15:36.106918 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35556def_a4de_4ccf_984f_7700074c610c.slice/crio-836274ef3990764502bf3c4f96be4073ea482659b53a637a692b84f733436a45 WatchSource:0}: Error finding container 836274ef3990764502bf3c4f96be4073ea482659b53a637a692b84f733436a45: Status 404 returned error can't find the container with id 836274ef3990764502bf3c4f96be4073ea482659b53a637a692b84f733436a45 Jul 08 22:15:36.753806 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:36.753762 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"35556def-a4de-4ccf-984f-7700074c610c","Type":"ContainerStarted","Data":"836274ef3990764502bf3c4f96be4073ea482659b53a637a692b84f733436a45"} Jul 08 22:15:37.760376 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:37.760339 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-thl2v" event={"ID":"f38983c4-b714-4da9-9336-28b1da2c78b5","Type":"ContainerStarted","Data":"d4bda236772db63cfe4d658934cae3fe13b6aa74de7cb71298a3054f4c8a4d09"} Jul 08 22:15:37.761138 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:37.760673 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-thl2v" Jul 08 22:15:37.762948 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:37.762918 2563 patch_prober.go:28] interesting pod/monitoring-plugin-d5dcff7cf-thl2v container/monitoring-plugin namespace/openshift-monitoring: Readiness probe status=failure output="Get \"https://10.134.0.14:9443/health\": dial tcp 10.134.0.14:9443: connect: connection refused" start-of-body= Jul 08 22:15:37.763088 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:37.762973 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-thl2v" podUID="f38983c4-b714-4da9-9336-28b1da2c78b5" containerName="monitoring-plugin" probeResult="failure" output="Get \"https://10.134.0.14:9443/health\": dial tcp 10.134.0.14:9443: connect: connection refused" Jul 08 22:15:37.763213 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:37.763184 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jhpj6" event={"ID":"7c5c6b58-e07d-4d72-af33-ebe169402d4e","Type":"ContainerStarted","Data":"f8760aa796dbbed096b2817e01595288676b56b64591b3b17526478802de6584"} Jul 08 22:15:37.779928 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:37.779461 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-thl2v" podStartSLOduration=1.346997674 podStartE2EDuration="3.779439421s" totalImagesPullingTime="2.432441747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 22:15:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:15:37.778328006 +0000 UTC m=+69.046348035" watchObservedRunningTime="2026-07-08 22:15:37.779439421 +0000 UTC m=+69.047459457" Jul 08 22:15:38.769932 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:38.769871 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jhpj6" event={"ID":"7c5c6b58-e07d-4d72-af33-ebe169402d4e","Type":"ContainerStarted","Data":"2faa656f92db69ea9cc39ad0f9d823958716b20c88e8fe383999c14f421e400e"} Jul 08 22:15:38.771760 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:38.771729 2563 generic.go:356] "Generic (PLEG): container finished" podID="35556def-a4de-4ccf-984f-7700074c610c" containerID="ea076cfbd8e0d0031cc9ff72c3fb76fcb4efaa03bfa0ae37ab614269507509a1" exitCode=0 Jul 08 22:15:38.771906 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:38.771819 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"35556def-a4de-4ccf-984f-7700074c610c","Type":"ContainerDied","Data":"ea076cfbd8e0d0031cc9ff72c3fb76fcb4efaa03bfa0ae37ab614269507509a1"} Jul 08 22:15:38.777751 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:38.777731 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-thl2v" Jul 08 22:15:38.786245 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:38.786190 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-jhpj6" podStartSLOduration=67.489103595 podStartE2EDuration="1m9.786168454s" totalImagesPullingTime="2.297064859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 22:14:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:15:38.785089748 +0000 UTC m=+70.053109789" watchObservedRunningTime="2026-07-08 22:15:38.786168454 +0000 UTC m=+70.054188483" Jul 08 22:15:40.159358 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:40.159315 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c5f6bd69f-2l4z7"] Jul 08 22:15:40.189782 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:40.189742 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c759449d4-gtwmk"] Jul 08 22:15:40.192274 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:40.192248 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c759449d4-gtwmk" Jul 08 22:15:40.206216 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:40.206108 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c759449d4-gtwmk"] Jul 08 22:15:40.312165 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:40.312122 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aed1b0ef-2682-43b4-af9f-2f3230ece91a-oauth-serving-cert\") pod \"console-5c759449d4-gtwmk\" (UID: \"aed1b0ef-2682-43b4-af9f-2f3230ece91a\") " pod="openshift-console/console-5c759449d4-gtwmk" Jul 08 22:15:40.312350 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:40.312199 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aed1b0ef-2682-43b4-af9f-2f3230ece91a-console-oauth-config\") pod \"console-5c759449d4-gtwmk\" (UID: \"aed1b0ef-2682-43b4-af9f-2f3230ece91a\") " pod="openshift-console/console-5c759449d4-gtwmk" Jul 08 22:15:40.312350 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:40.312232 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rmld\" (UniqueName: \"kubernetes.io/projected/aed1b0ef-2682-43b4-af9f-2f3230ece91a-kube-api-access-2rmld\") pod \"console-5c759449d4-gtwmk\" (UID: \"aed1b0ef-2682-43b4-af9f-2f3230ece91a\") " pod="openshift-console/console-5c759449d4-gtwmk" Jul 08 22:15:40.312350 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:40.312264 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aed1b0ef-2682-43b4-af9f-2f3230ece91a-console-serving-cert\") pod \"console-5c759449d4-gtwmk\" (UID: \"aed1b0ef-2682-43b4-af9f-2f3230ece91a\") " pod="openshift-console/console-5c759449d4-gtwmk" Jul 08 22:15:40.312350 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:40.312292 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aed1b0ef-2682-43b4-af9f-2f3230ece91a-trusted-ca-bundle\") pod \"console-5c759449d4-gtwmk\" (UID: \"aed1b0ef-2682-43b4-af9f-2f3230ece91a\") " pod="openshift-console/console-5c759449d4-gtwmk" Jul 08 22:15:40.312516 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:40.312372 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aed1b0ef-2682-43b4-af9f-2f3230ece91a-service-ca\") pod \"console-5c759449d4-gtwmk\" (UID: \"aed1b0ef-2682-43b4-af9f-2f3230ece91a\") " pod="openshift-console/console-5c759449d4-gtwmk" Jul 08 22:15:40.312516 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:40.312427 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aed1b0ef-2682-43b4-af9f-2f3230ece91a-console-config\") pod \"console-5c759449d4-gtwmk\" (UID: \"aed1b0ef-2682-43b4-af9f-2f3230ece91a\") " pod="openshift-console/console-5c759449d4-gtwmk" Jul 08 22:15:40.416139 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:40.413477 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aed1b0ef-2682-43b4-af9f-2f3230ece91a-console-serving-cert\") pod \"console-5c759449d4-gtwmk\" (UID: \"aed1b0ef-2682-43b4-af9f-2f3230ece91a\") " pod="openshift-console/console-5c759449d4-gtwmk" Jul 08 22:15:40.416139 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:40.413520 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aed1b0ef-2682-43b4-af9f-2f3230ece91a-trusted-ca-bundle\") pod \"console-5c759449d4-gtwmk\" (UID: \"aed1b0ef-2682-43b4-af9f-2f3230ece91a\") " pod="openshift-console/console-5c759449d4-gtwmk" Jul 08 22:15:40.416139 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:40.413565 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aed1b0ef-2682-43b4-af9f-2f3230ece91a-service-ca\") pod \"console-5c759449d4-gtwmk\" (UID: \"aed1b0ef-2682-43b4-af9f-2f3230ece91a\") " pod="openshift-console/console-5c759449d4-gtwmk" Jul 08 22:15:40.416139 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:40.413606 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aed1b0ef-2682-43b4-af9f-2f3230ece91a-console-config\") pod \"console-5c759449d4-gtwmk\" (UID: \"aed1b0ef-2682-43b4-af9f-2f3230ece91a\") " pod="openshift-console/console-5c759449d4-gtwmk" Jul 08 22:15:40.416139 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:40.413668 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aed1b0ef-2682-43b4-af9f-2f3230ece91a-oauth-serving-cert\") pod \"console-5c759449d4-gtwmk\" (UID: \"aed1b0ef-2682-43b4-af9f-2f3230ece91a\") " pod="openshift-console/console-5c759449d4-gtwmk" Jul 08 22:15:40.416139 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:40.413704 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aed1b0ef-2682-43b4-af9f-2f3230ece91a-console-oauth-config\") pod \"console-5c759449d4-gtwmk\" (UID: \"aed1b0ef-2682-43b4-af9f-2f3230ece91a\") " pod="openshift-console/console-5c759449d4-gtwmk" Jul 08 22:15:40.416139 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:40.413737 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rmld\" (UniqueName: \"kubernetes.io/projected/aed1b0ef-2682-43b4-af9f-2f3230ece91a-kube-api-access-2rmld\") pod \"console-5c759449d4-gtwmk\" (UID: \"aed1b0ef-2682-43b4-af9f-2f3230ece91a\") " pod="openshift-console/console-5c759449d4-gtwmk" Jul 08 22:15:40.416139 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:40.415221 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aed1b0ef-2682-43b4-af9f-2f3230ece91a-console-config\") pod \"console-5c759449d4-gtwmk\" (UID: \"aed1b0ef-2682-43b4-af9f-2f3230ece91a\") " pod="openshift-console/console-5c759449d4-gtwmk" Jul 08 22:15:40.416139 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:40.415977 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aed1b0ef-2682-43b4-af9f-2f3230ece91a-trusted-ca-bundle\") pod \"console-5c759449d4-gtwmk\" (UID: \"aed1b0ef-2682-43b4-af9f-2f3230ece91a\") " pod="openshift-console/console-5c759449d4-gtwmk" Jul 08 22:15:40.416766 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:40.416549 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aed1b0ef-2682-43b4-af9f-2f3230ece91a-service-ca\") pod \"console-5c759449d4-gtwmk\" (UID: \"aed1b0ef-2682-43b4-af9f-2f3230ece91a\") " pod="openshift-console/console-5c759449d4-gtwmk" Jul 08 22:15:40.417124 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:40.417095 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aed1b0ef-2682-43b4-af9f-2f3230ece91a-oauth-serving-cert\") pod \"console-5c759449d4-gtwmk\" (UID: \"aed1b0ef-2682-43b4-af9f-2f3230ece91a\") " pod="openshift-console/console-5c759449d4-gtwmk" Jul 08 22:15:40.417642 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:40.417616 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aed1b0ef-2682-43b4-af9f-2f3230ece91a-console-serving-cert\") pod \"console-5c759449d4-gtwmk\" (UID: \"aed1b0ef-2682-43b4-af9f-2f3230ece91a\") " pod="openshift-console/console-5c759449d4-gtwmk" Jul 08 22:15:40.419144 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:40.419117 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aed1b0ef-2682-43b4-af9f-2f3230ece91a-console-oauth-config\") pod \"console-5c759449d4-gtwmk\" (UID: \"aed1b0ef-2682-43b4-af9f-2f3230ece91a\") " pod="openshift-console/console-5c759449d4-gtwmk" Jul 08 22:15:40.422741 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:40.422715 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rmld\" (UniqueName: \"kubernetes.io/projected/aed1b0ef-2682-43b4-af9f-2f3230ece91a-kube-api-access-2rmld\") pod \"console-5c759449d4-gtwmk\" (UID: \"aed1b0ef-2682-43b4-af9f-2f3230ece91a\") " pod="openshift-console/console-5c759449d4-gtwmk" Jul 08 22:15:40.503218 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:40.503183 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c759449d4-gtwmk" Jul 08 22:15:41.324127 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:41.324091 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c759449d4-gtwmk"] Jul 08 22:15:41.328093 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:15:41.328071 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaed1b0ef_2682_43b4_af9f_2f3230ece91a.slice/crio-017b840a1fab01f332fd05468bbdff84a731a4bf8be2278ffe3e00331b0471a7 WatchSource:0}: Error finding container 017b840a1fab01f332fd05468bbdff84a731a4bf8be2278ffe3e00331b0471a7: Status 404 returned error can't find the container with id 017b840a1fab01f332fd05468bbdff84a731a4bf8be2278ffe3e00331b0471a7 Jul 08 22:15:41.788656 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:41.788308 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-98x2n" event={"ID":"6a151365-c622-4361-862f-ed202f3045f7","Type":"ContainerStarted","Data":"972491a6f3f3957efc637b99cb68164ae19488402449b3a74d567328596fa334"} Jul 08 22:15:41.788656 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:41.788438 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:15:41.790054 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:41.790017 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fffds" event={"ID":"4a354dc2-56c0-4280-96d1-b560d70be2b2","Type":"ContainerStarted","Data":"a694d250792c9ed6b9fb3ac00b99bd9c1dd3f6c23e75f97053639ebb4dee3f56"} Jul 08 22:15:41.791795 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:41.791742 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c759449d4-gtwmk" event={"ID":"aed1b0ef-2682-43b4-af9f-2f3230ece91a","Type":"ContainerStarted","Data":"dbfeba2570431e9a11ca197565b50166598c98f7807c8ae14054dc3e7cf3fd32"} Jul 08 22:15:41.791795 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:41.791790 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c759449d4-gtwmk" event={"ID":"aed1b0ef-2682-43b4-af9f-2f3230ece91a","Type":"ContainerStarted","Data":"017b840a1fab01f332fd05468bbdff84a731a4bf8be2278ffe3e00331b0471a7"} Jul 08 22:15:41.804342 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:41.804294 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-98x2n" podStartSLOduration=67.261109588 podStartE2EDuration="1m12.804278519s" totalImagesPullingTime="5.543168931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 22:14:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:15:41.803762899 +0000 UTC m=+73.071782929" watchObservedRunningTime="2026-07-08 22:15:41.804278519 +0000 UTC m=+73.072298548" Jul 08 22:15:41.821666 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:41.821613 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5c759449d4-gtwmk" podStartSLOduration=1.821596903 podStartE2EDuration="1.821596903s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 22:15:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:15:41.820753871 +0000 UTC m=+73.088773901" watchObservedRunningTime="2026-07-08 22:15:41.821596903 +0000 UTC m=+73.089616931" Jul 08 22:15:41.835695 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:41.835647 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-fffds" podStartSLOduration=66.950662025 podStartE2EDuration="1m12.835629186s" totalImagesPullingTime="5.884967161s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 22:14:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:15:41.83454572 +0000 UTC m=+73.102565750" watchObservedRunningTime="2026-07-08 22:15:41.835629186 +0000 UTC m=+73.103649214" Jul 08 22:15:44.506115 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:44.506075 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c4585cb94-tw6td" Jul 08 22:15:44.506539 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:44.506142 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c4585cb94-tw6td" Jul 08 22:15:44.511878 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:44.511852 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c4585cb94-tw6td" Jul 08 22:15:44.807922 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:44.807868 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"35556def-a4de-4ccf-984f-7700074c610c","Type":"ContainerStarted","Data":"f02f34d86a139d7519b22fea82b13f95dd949c23fdcb75fd8136b1fdce54f7f4"} Jul 08 22:15:44.808111 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:44.807971 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"35556def-a4de-4ccf-984f-7700074c610c","Type":"ContainerStarted","Data":"2a017ef95710dc8de2a3946e157b9a07080bc59c5ad5f02d1005bb710b45d7b8"} Jul 08 22:15:44.812837 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:44.812809 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c4585cb94-tw6td" Jul 08 22:15:46.820432 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:46.820392 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"35556def-a4de-4ccf-984f-7700074c610c","Type":"ContainerStarted","Data":"31f993706c163fefd47d617879ea96a1d5abe225aea9397c013ffdb0091d0f5d"} Jul 08 22:15:46.820763 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:46.820488 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"35556def-a4de-4ccf-984f-7700074c610c","Type":"ContainerStarted","Data":"4bfba3ee8d60dc9314129bd39e1b86cbcbc83ebf236b564445829d927908d108"} Jul 08 22:15:46.820763 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:46.820516 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"35556def-a4de-4ccf-984f-7700074c610c","Type":"ContainerStarted","Data":"ffe5f3b2d2d8471db7d401688f7a255314d15513f6e2cc336f433bda3baba389"} Jul 08 22:15:47.828623 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:47.828586 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"35556def-a4de-4ccf-984f-7700074c610c","Type":"ContainerStarted","Data":"9e81381c60d659d4968272830a489c51bd947a116e5c7944656bfe9a81c3bfa3"} Jul 08 22:15:47.863104 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:47.863048 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.2556768050000002 podStartE2EDuration="12.863026725s" totalImagesPullingTime="9.579390454s" totalInitContainerRuntime="27.959466ms" isStatefulPod=false podCreationTimestamp="2026-07-08 22:15:35 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:15:47.86079031 +0000 UTC m=+79.128810339" watchObservedRunningTime="2026-07-08 22:15:47.863026725 +0000 UTC m=+79.131046755" Jul 08 22:15:50.503973 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:50.503828 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c759449d4-gtwmk" Jul 08 22:15:50.503973 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:50.503916 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c759449d4-gtwmk" Jul 08 22:15:50.509084 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:50.509060 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c759449d4-gtwmk" Jul 08 22:15:50.845686 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:50.845604 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c759449d4-gtwmk" Jul 08 22:15:50.891605 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:50.891571 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c4585cb94-tw6td"] Jul 08 22:15:50.894380 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:50.894348 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:15:59.179623 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:59.179576 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-658dc76d8d-lx2hh" podUID="28e04205-7edb-4465-928d-ef26a27ae65c" containerName="console" containerID="cri-o://f33b42f41e37533e0861ef2d7ba3ba63bb138374d9b60c37b98087a0a9679c49" gracePeriod=15 Jul 08 22:15:59.878989 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:59.878874 2563 generic.go:356] "Generic (PLEG): container finished" podID="28e04205-7edb-4465-928d-ef26a27ae65c" containerID="f33b42f41e37533e0861ef2d7ba3ba63bb138374d9b60c37b98087a0a9679c49" exitCode=0 Jul 08 22:15:59.878989 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:59.878962 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-658dc76d8d-lx2hh" event={"ID":"28e04205-7edb-4465-928d-ef26a27ae65c","Type":"ContainerDied","Data":"f33b42f41e37533e0861ef2d7ba3ba63bb138374d9b60c37b98087a0a9679c49"} Jul 08 22:15:59.991481 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:15:59.991445 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-658dc76d8d-lx2hh" Jul 08 22:16:00.095876 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:00.095785 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/28e04205-7edb-4465-928d-ef26a27ae65c-console-config\") pod \"28e04205-7edb-4465-928d-ef26a27ae65c\" (UID: \"28e04205-7edb-4465-928d-ef26a27ae65c\") " Jul 08 22:16:00.095876 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:00.095845 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/28e04205-7edb-4465-928d-ef26a27ae65c-console-serving-cert\") pod \"28e04205-7edb-4465-928d-ef26a27ae65c\" (UID: \"28e04205-7edb-4465-928d-ef26a27ae65c\") " Jul 08 22:16:00.096131 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:00.095900 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/28e04205-7edb-4465-928d-ef26a27ae65c-service-ca\") pod \"28e04205-7edb-4465-928d-ef26a27ae65c\" (UID: \"28e04205-7edb-4465-928d-ef26a27ae65c\") " Jul 08 22:16:00.096131 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:00.095931 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/28e04205-7edb-4465-928d-ef26a27ae65c-console-oauth-config\") pod \"28e04205-7edb-4465-928d-ef26a27ae65c\" (UID: \"28e04205-7edb-4465-928d-ef26a27ae65c\") " Jul 08 22:16:00.096131 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:00.095973 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/28e04205-7edb-4465-928d-ef26a27ae65c-oauth-serving-cert\") pod \"28e04205-7edb-4465-928d-ef26a27ae65c\" (UID: \"28e04205-7edb-4465-928d-ef26a27ae65c\") " Jul 08 22:16:00.096131 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:00.096001 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nwnhh\" (UniqueName: \"kubernetes.io/projected/28e04205-7edb-4465-928d-ef26a27ae65c-kube-api-access-nwnhh\") pod \"28e04205-7edb-4465-928d-ef26a27ae65c\" (UID: \"28e04205-7edb-4465-928d-ef26a27ae65c\") " Jul 08 22:16:00.096325 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:00.096226 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28e04205-7edb-4465-928d-ef26a27ae65c-service-ca" (OuterVolumeSpecName: "service-ca") pod "28e04205-7edb-4465-928d-ef26a27ae65c" (UID: "28e04205-7edb-4465-928d-ef26a27ae65c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 22:16:00.096325 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:00.096295 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28e04205-7edb-4465-928d-ef26a27ae65c-console-config" (OuterVolumeSpecName: "console-config") pod "28e04205-7edb-4465-928d-ef26a27ae65c" (UID: "28e04205-7edb-4465-928d-ef26a27ae65c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 22:16:00.096439 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:00.096346 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28e04205-7edb-4465-928d-ef26a27ae65c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "28e04205-7edb-4465-928d-ef26a27ae65c" (UID: "28e04205-7edb-4465-928d-ef26a27ae65c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 22:16:00.098383 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:00.098354 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28e04205-7edb-4465-928d-ef26a27ae65c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "28e04205-7edb-4465-928d-ef26a27ae65c" (UID: "28e04205-7edb-4465-928d-ef26a27ae65c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 22:16:00.098504 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:00.098449 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28e04205-7edb-4465-928d-ef26a27ae65c-kube-api-access-nwnhh" (OuterVolumeSpecName: "kube-api-access-nwnhh") pod "28e04205-7edb-4465-928d-ef26a27ae65c" (UID: "28e04205-7edb-4465-928d-ef26a27ae65c"). InnerVolumeSpecName "kube-api-access-nwnhh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 22:16:00.098504 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:00.098448 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28e04205-7edb-4465-928d-ef26a27ae65c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "28e04205-7edb-4465-928d-ef26a27ae65c" (UID: "28e04205-7edb-4465-928d-ef26a27ae65c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 22:16:00.197098 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:00.197058 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/28e04205-7edb-4465-928d-ef26a27ae65c-oauth-serving-cert\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:00.197098 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:00.197090 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nwnhh\" (UniqueName: \"kubernetes.io/projected/28e04205-7edb-4465-928d-ef26a27ae65c-kube-api-access-nwnhh\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:00.197098 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:00.197100 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/28e04205-7edb-4465-928d-ef26a27ae65c-console-config\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:00.197098 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:00.197108 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/28e04205-7edb-4465-928d-ef26a27ae65c-console-serving-cert\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:00.197565 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:00.197116 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/28e04205-7edb-4465-928d-ef26a27ae65c-service-ca\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:00.197565 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:00.197126 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/28e04205-7edb-4465-928d-ef26a27ae65c-console-oauth-config\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:00.884077 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:00.884051 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-658dc76d8d-lx2hh" Jul 08 22:16:00.884279 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:00.884044 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-658dc76d8d-lx2hh" event={"ID":"28e04205-7edb-4465-928d-ef26a27ae65c","Type":"ContainerDied","Data":"fba51e140bc6f62468f2d342a13b7b8aee547998e98a99e88a140248a53f270e"} Jul 08 22:16:00.884279 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:00.884187 2563 scope.go:117] "RemoveContainer" containerID="f33b42f41e37533e0861ef2d7ba3ba63bb138374d9b60c37b98087a0a9679c49" Jul 08 22:16:00.905438 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:00.905406 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-658dc76d8d-lx2hh"] Jul 08 22:16:00.909037 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:00.909011 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-658dc76d8d-lx2hh"] Jul 08 22:16:01.317307 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:01.317274 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28e04205-7edb-4465-928d-ef26a27ae65c" path="/var/lib/kubelet/pods/28e04205-7edb-4465-928d-ef26a27ae65c/volumes" Jul 08 22:16:05.183343 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:05.183278 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5c5f6bd69f-2l4z7" podUID="5d62c766-320a-4fea-ac5a-23db8702e521" containerName="console" containerID="cri-o://0faa4e6d1326ebaa4a099fa9d99917ba21fce14ceba5f75be33e005aa2383ec6" gracePeriod=15 Jul 08 22:16:05.423736 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:05.423713 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5f6bd69f-2l4z7" Jul 08 22:16:05.436850 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:05.436782 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d62c766-320a-4fea-ac5a-23db8702e521-trusted-ca-bundle\") pod \"5d62c766-320a-4fea-ac5a-23db8702e521\" (UID: \"5d62c766-320a-4fea-ac5a-23db8702e521\") " Jul 08 22:16:05.436850 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:05.436820 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5d62c766-320a-4fea-ac5a-23db8702e521-oauth-serving-cert\") pod \"5d62c766-320a-4fea-ac5a-23db8702e521\" (UID: \"5d62c766-320a-4fea-ac5a-23db8702e521\") " Jul 08 22:16:05.436850 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:05.436842 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bgvzk\" (UniqueName: \"kubernetes.io/projected/5d62c766-320a-4fea-ac5a-23db8702e521-kube-api-access-bgvzk\") pod \"5d62c766-320a-4fea-ac5a-23db8702e521\" (UID: \"5d62c766-320a-4fea-ac5a-23db8702e521\") " Jul 08 22:16:05.437067 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:05.436938 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5d62c766-320a-4fea-ac5a-23db8702e521-console-config\") pod \"5d62c766-320a-4fea-ac5a-23db8702e521\" (UID: \"5d62c766-320a-4fea-ac5a-23db8702e521\") " Jul 08 22:16:05.437067 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:05.436966 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5d62c766-320a-4fea-ac5a-23db8702e521-service-ca\") pod \"5d62c766-320a-4fea-ac5a-23db8702e521\" (UID: \"5d62c766-320a-4fea-ac5a-23db8702e521\") " Jul 08 22:16:05.437067 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:05.437005 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5d62c766-320a-4fea-ac5a-23db8702e521-console-serving-cert\") pod \"5d62c766-320a-4fea-ac5a-23db8702e521\" (UID: \"5d62c766-320a-4fea-ac5a-23db8702e521\") " Jul 08 22:16:05.437067 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:05.437037 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5d62c766-320a-4fea-ac5a-23db8702e521-console-oauth-config\") pod \"5d62c766-320a-4fea-ac5a-23db8702e521\" (UID: \"5d62c766-320a-4fea-ac5a-23db8702e521\") " Jul 08 22:16:05.437359 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:05.437314 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d62c766-320a-4fea-ac5a-23db8702e521-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5d62c766-320a-4fea-ac5a-23db8702e521" (UID: "5d62c766-320a-4fea-ac5a-23db8702e521"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 22:16:05.437434 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:05.437404 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d62c766-320a-4fea-ac5a-23db8702e521-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5d62c766-320a-4fea-ac5a-23db8702e521" (UID: "5d62c766-320a-4fea-ac5a-23db8702e521"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 22:16:05.437434 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:05.437409 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d62c766-320a-4fea-ac5a-23db8702e521-console-config" (OuterVolumeSpecName: "console-config") pod "5d62c766-320a-4fea-ac5a-23db8702e521" (UID: "5d62c766-320a-4fea-ac5a-23db8702e521"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 22:16:05.437562 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:05.437539 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d62c766-320a-4fea-ac5a-23db8702e521-service-ca" (OuterVolumeSpecName: "service-ca") pod "5d62c766-320a-4fea-ac5a-23db8702e521" (UID: "5d62c766-320a-4fea-ac5a-23db8702e521"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 22:16:05.439182 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:05.439151 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d62c766-320a-4fea-ac5a-23db8702e521-kube-api-access-bgvzk" (OuterVolumeSpecName: "kube-api-access-bgvzk") pod "5d62c766-320a-4fea-ac5a-23db8702e521" (UID: "5d62c766-320a-4fea-ac5a-23db8702e521"). InnerVolumeSpecName "kube-api-access-bgvzk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 22:16:05.439290 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:05.439236 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d62c766-320a-4fea-ac5a-23db8702e521-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5d62c766-320a-4fea-ac5a-23db8702e521" (UID: "5d62c766-320a-4fea-ac5a-23db8702e521"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 22:16:05.439333 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:05.439290 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d62c766-320a-4fea-ac5a-23db8702e521-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5d62c766-320a-4fea-ac5a-23db8702e521" (UID: "5d62c766-320a-4fea-ac5a-23db8702e521"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 22:16:05.538909 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:05.538840 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5d62c766-320a-4fea-ac5a-23db8702e521-console-config\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:05.538909 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:05.538875 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5d62c766-320a-4fea-ac5a-23db8702e521-service-ca\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:05.538909 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:05.538913 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5d62c766-320a-4fea-ac5a-23db8702e521-console-serving-cert\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:05.539155 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:05.538923 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5d62c766-320a-4fea-ac5a-23db8702e521-console-oauth-config\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:05.539155 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:05.538934 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d62c766-320a-4fea-ac5a-23db8702e521-trusted-ca-bundle\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:05.539155 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:05.538943 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5d62c766-320a-4fea-ac5a-23db8702e521-oauth-serving-cert\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:05.539155 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:05.538952 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgvzk\" (UniqueName: \"kubernetes.io/projected/5d62c766-320a-4fea-ac5a-23db8702e521-kube-api-access-bgvzk\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:05.906552 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:05.906513 2563 generic.go:356] "Generic (PLEG): container finished" podID="5d62c766-320a-4fea-ac5a-23db8702e521" containerID="0faa4e6d1326ebaa4a099fa9d99917ba21fce14ceba5f75be33e005aa2383ec6" exitCode=0 Jul 08 22:16:05.906727 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:05.906570 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5f6bd69f-2l4z7" Jul 08 22:16:05.906727 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:05.906603 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5f6bd69f-2l4z7" event={"ID":"5d62c766-320a-4fea-ac5a-23db8702e521","Type":"ContainerDied","Data":"0faa4e6d1326ebaa4a099fa9d99917ba21fce14ceba5f75be33e005aa2383ec6"} Jul 08 22:16:05.906727 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:05.906657 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5f6bd69f-2l4z7" event={"ID":"5d62c766-320a-4fea-ac5a-23db8702e521","Type":"ContainerDied","Data":"63ddf2cdc2f2ad42cdf12002f3bae7e3e54aedcf2e38cc459528dcfcf6e61cab"} Jul 08 22:16:05.906727 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:05.906679 2563 scope.go:117] "RemoveContainer" containerID="0faa4e6d1326ebaa4a099fa9d99917ba21fce14ceba5f75be33e005aa2383ec6" Jul 08 22:16:05.915539 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:05.915519 2563 scope.go:117] "RemoveContainer" containerID="0faa4e6d1326ebaa4a099fa9d99917ba21fce14ceba5f75be33e005aa2383ec6" Jul 08 22:16:05.915869 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:16:05.915848 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0faa4e6d1326ebaa4a099fa9d99917ba21fce14ceba5f75be33e005aa2383ec6\": container with ID starting with 0faa4e6d1326ebaa4a099fa9d99917ba21fce14ceba5f75be33e005aa2383ec6 not found: ID does not exist" containerID="0faa4e6d1326ebaa4a099fa9d99917ba21fce14ceba5f75be33e005aa2383ec6" Jul 08 22:16:05.915943 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:05.915903 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0faa4e6d1326ebaa4a099fa9d99917ba21fce14ceba5f75be33e005aa2383ec6"} err="failed to get container status \"0faa4e6d1326ebaa4a099fa9d99917ba21fce14ceba5f75be33e005aa2383ec6\": rpc error: code = NotFound desc = could not find container \"0faa4e6d1326ebaa4a099fa9d99917ba21fce14ceba5f75be33e005aa2383ec6\": container with ID starting with 0faa4e6d1326ebaa4a099fa9d99917ba21fce14ceba5f75be33e005aa2383ec6 not found: ID does not exist" Jul 08 22:16:05.928434 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:05.928368 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c5f6bd69f-2l4z7"] Jul 08 22:16:05.930066 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:05.930034 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5c5f6bd69f-2l4z7"] Jul 08 22:16:07.317142 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:07.317106 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d62c766-320a-4fea-ac5a-23db8702e521" path="/var/lib/kubelet/pods/5d62c766-320a-4fea-ac5a-23db8702e521/volumes" Jul 08 22:16:12.798330 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:12.798292 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-98x2n" Jul 08 22:16:15.921033 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:15.920994 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7c4585cb94-tw6td" podUID="6c022be9-ee23-4a42-b174-f502add283f7" containerName="console" containerID="cri-o://007e9c5d458a47d85cb82352fca69e084c53da2b21aa1dc9151de85f0f5e5577" gracePeriod=15 Jul 08 22:16:16.204527 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:16.204495 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c4585cb94-tw6td" Jul 08 22:16:16.342868 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:16.342826 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6c022be9-ee23-4a42-b174-f502add283f7-service-ca\") pod \"6c022be9-ee23-4a42-b174-f502add283f7\" (UID: \"6c022be9-ee23-4a42-b174-f502add283f7\") " Jul 08 22:16:16.342868 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:16.342866 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6c022be9-ee23-4a42-b174-f502add283f7-console-config\") pod \"6c022be9-ee23-4a42-b174-f502add283f7\" (UID: \"6c022be9-ee23-4a42-b174-f502add283f7\") " Jul 08 22:16:16.343149 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:16.342929 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p6knq\" (UniqueName: \"kubernetes.io/projected/6c022be9-ee23-4a42-b174-f502add283f7-kube-api-access-p6knq\") pod \"6c022be9-ee23-4a42-b174-f502add283f7\" (UID: \"6c022be9-ee23-4a42-b174-f502add283f7\") " Jul 08 22:16:16.343149 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:16.342952 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6c022be9-ee23-4a42-b174-f502add283f7-oauth-serving-cert\") pod \"6c022be9-ee23-4a42-b174-f502add283f7\" (UID: \"6c022be9-ee23-4a42-b174-f502add283f7\") " Jul 08 22:16:16.343149 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:16.343021 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6c022be9-ee23-4a42-b174-f502add283f7-console-serving-cert\") pod \"6c022be9-ee23-4a42-b174-f502add283f7\" (UID: \"6c022be9-ee23-4a42-b174-f502add283f7\") " Jul 08 22:16:16.343149 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:16.343046 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6c022be9-ee23-4a42-b174-f502add283f7-console-oauth-config\") pod \"6c022be9-ee23-4a42-b174-f502add283f7\" (UID: \"6c022be9-ee23-4a42-b174-f502add283f7\") " Jul 08 22:16:16.343149 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:16.343076 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c022be9-ee23-4a42-b174-f502add283f7-trusted-ca-bundle\") pod \"6c022be9-ee23-4a42-b174-f502add283f7\" (UID: \"6c022be9-ee23-4a42-b174-f502add283f7\") " Jul 08 22:16:16.343443 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:16.343318 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c022be9-ee23-4a42-b174-f502add283f7-service-ca" (OuterVolumeSpecName: "service-ca") pod "6c022be9-ee23-4a42-b174-f502add283f7" (UID: "6c022be9-ee23-4a42-b174-f502add283f7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 22:16:16.343485 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:16.343419 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c022be9-ee23-4a42-b174-f502add283f7-console-config" (OuterVolumeSpecName: "console-config") pod "6c022be9-ee23-4a42-b174-f502add283f7" (UID: "6c022be9-ee23-4a42-b174-f502add283f7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 22:16:16.343525 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:16.343502 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c022be9-ee23-4a42-b174-f502add283f7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6c022be9-ee23-4a42-b174-f502add283f7" (UID: "6c022be9-ee23-4a42-b174-f502add283f7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 22:16:16.343731 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:16.343709 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c022be9-ee23-4a42-b174-f502add283f7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6c022be9-ee23-4a42-b174-f502add283f7" (UID: "6c022be9-ee23-4a42-b174-f502add283f7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 22:16:16.345420 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:16.345389 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c022be9-ee23-4a42-b174-f502add283f7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6c022be9-ee23-4a42-b174-f502add283f7" (UID: "6c022be9-ee23-4a42-b174-f502add283f7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 22:16:16.345534 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:16.345424 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c022be9-ee23-4a42-b174-f502add283f7-kube-api-access-p6knq" (OuterVolumeSpecName: "kube-api-access-p6knq") pod "6c022be9-ee23-4a42-b174-f502add283f7" (UID: "6c022be9-ee23-4a42-b174-f502add283f7"). InnerVolumeSpecName "kube-api-access-p6knq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 22:16:16.345534 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:16.345453 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c022be9-ee23-4a42-b174-f502add283f7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6c022be9-ee23-4a42-b174-f502add283f7" (UID: "6c022be9-ee23-4a42-b174-f502add283f7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 22:16:16.444010 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:16.443918 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6c022be9-ee23-4a42-b174-f502add283f7-console-serving-cert\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:16.444010 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:16.443951 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6c022be9-ee23-4a42-b174-f502add283f7-console-oauth-config\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:16.444010 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:16.443961 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c022be9-ee23-4a42-b174-f502add283f7-trusted-ca-bundle\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:16.444010 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:16.443970 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6c022be9-ee23-4a42-b174-f502add283f7-service-ca\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:16.444010 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:16.443979 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6c022be9-ee23-4a42-b174-f502add283f7-console-config\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:16.444010 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:16.443987 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p6knq\" (UniqueName: \"kubernetes.io/projected/6c022be9-ee23-4a42-b174-f502add283f7-kube-api-access-p6knq\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:16.444010 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:16.443995 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6c022be9-ee23-4a42-b174-f502add283f7-oauth-serving-cert\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:16.954834 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:16.954792 2563 generic.go:356] "Generic (PLEG): container finished" podID="6c022be9-ee23-4a42-b174-f502add283f7" containerID="007e9c5d458a47d85cb82352fca69e084c53da2b21aa1dc9151de85f0f5e5577" exitCode=0 Jul 08 22:16:16.955245 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:16.954901 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c4585cb94-tw6td" Jul 08 22:16:16.955245 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:16.954897 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c4585cb94-tw6td" event={"ID":"6c022be9-ee23-4a42-b174-f502add283f7","Type":"ContainerDied","Data":"007e9c5d458a47d85cb82352fca69e084c53da2b21aa1dc9151de85f0f5e5577"} Jul 08 22:16:16.955245 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:16.955044 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c4585cb94-tw6td" event={"ID":"6c022be9-ee23-4a42-b174-f502add283f7","Type":"ContainerDied","Data":"0087bbc1d942f50f5721ec6ecbb91f4912df245f135de9f6b365302ad95ea091"} Jul 08 22:16:16.955245 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:16.955069 2563 scope.go:117] "RemoveContainer" containerID="007e9c5d458a47d85cb82352fca69e084c53da2b21aa1dc9151de85f0f5e5577" Jul 08 22:16:16.963955 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:16.963808 2563 scope.go:117] "RemoveContainer" containerID="007e9c5d458a47d85cb82352fca69e084c53da2b21aa1dc9151de85f0f5e5577" Jul 08 22:16:16.964201 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:16:16.964168 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"007e9c5d458a47d85cb82352fca69e084c53da2b21aa1dc9151de85f0f5e5577\": container with ID starting with 007e9c5d458a47d85cb82352fca69e084c53da2b21aa1dc9151de85f0f5e5577 not found: ID does not exist" containerID="007e9c5d458a47d85cb82352fca69e084c53da2b21aa1dc9151de85f0f5e5577" Jul 08 22:16:16.964281 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:16.964210 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"007e9c5d458a47d85cb82352fca69e084c53da2b21aa1dc9151de85f0f5e5577"} err="failed to get container status \"007e9c5d458a47d85cb82352fca69e084c53da2b21aa1dc9151de85f0f5e5577\": rpc error: code = NotFound desc = could not find container \"007e9c5d458a47d85cb82352fca69e084c53da2b21aa1dc9151de85f0f5e5577\": container with ID starting with 007e9c5d458a47d85cb82352fca69e084c53da2b21aa1dc9151de85f0f5e5577 not found: ID does not exist" Jul 08 22:16:16.978513 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:16.978479 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c4585cb94-tw6td"] Jul 08 22:16:16.981111 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:16.980871 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c4585cb94-tw6td"] Jul 08 22:16:17.318286 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:17.318244 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c022be9-ee23-4a42-b174-f502add283f7" path="/var/lib/kubelet/pods/6c022be9-ee23-4a42-b174-f502add283f7/volumes" Jul 08 22:16:35.894318 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:35.894276 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:35.914766 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:35.914736 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:36.050052 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:36.050024 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:53.992103 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:53.992064 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 22:16:53.992697 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:53.992460 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="35556def-a4de-4ccf-984f-7700074c610c" containerName="prometheus" containerID="cri-o://2a017ef95710dc8de2a3946e157b9a07080bc59c5ad5f02d1005bb710b45d7b8" gracePeriod=600 Jul 08 22:16:53.992697 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:53.992514 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="35556def-a4de-4ccf-984f-7700074c610c" containerName="kube-rbac-proxy-thanos" containerID="cri-o://9e81381c60d659d4968272830a489c51bd947a116e5c7944656bfe9a81c3bfa3" gracePeriod=600 Jul 08 22:16:53.992697 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:53.992652 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="35556def-a4de-4ccf-984f-7700074c610c" containerName="kube-rbac-proxy" containerID="cri-o://31f993706c163fefd47d617879ea96a1d5abe225aea9397c013ffdb0091d0f5d" gracePeriod=600 Jul 08 22:16:53.992915 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:53.992758 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="35556def-a4de-4ccf-984f-7700074c610c" containerName="config-reloader" containerID="cri-o://f02f34d86a139d7519b22fea82b13f95dd949c23fdcb75fd8136b1fdce54f7f4" gracePeriod=600 Jul 08 22:16:53.992915 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:53.992768 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="35556def-a4de-4ccf-984f-7700074c610c" containerName="thanos-sidecar" containerID="cri-o://ffe5f3b2d2d8471db7d401688f7a255314d15513f6e2cc336f433bda3baba389" gracePeriod=600 Jul 08 22:16:53.992915 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:53.992878 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="35556def-a4de-4ccf-984f-7700074c610c" containerName="kube-rbac-proxy-web" containerID="cri-o://4bfba3ee8d60dc9314129bd39e1b86cbcbc83ebf236b564445829d927908d108" gracePeriod=600 Jul 08 22:16:54.259389 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.259365 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:54.350050 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.350020 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"35556def-a4de-4ccf-984f-7700074c610c\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " Jul 08 22:16:54.350050 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.350055 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-prometheus-k8s-rulefiles-2\") pod \"35556def-a4de-4ccf-984f-7700074c610c\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " Jul 08 22:16:54.350301 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.350075 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-secret-metrics-client-certs\") pod \"35556def-a4de-4ccf-984f-7700074c610c\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " Jul 08 22:16:54.350301 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.350093 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/35556def-a4de-4ccf-984f-7700074c610c-config-out\") pod \"35556def-a4de-4ccf-984f-7700074c610c\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " Jul 08 22:16:54.350301 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.350114 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m6njg\" (UniqueName: \"kubernetes.io/projected/35556def-a4de-4ccf-984f-7700074c610c-kube-api-access-m6njg\") pod \"35556def-a4de-4ccf-984f-7700074c610c\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " Jul 08 22:16:54.350301 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.350129 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-secret-grpc-tls\") pod \"35556def-a4de-4ccf-984f-7700074c610c\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " Jul 08 22:16:54.350301 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.350147 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-config\") pod \"35556def-a4de-4ccf-984f-7700074c610c\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " Jul 08 22:16:54.350301 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.350179 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-prometheus-k8s-rulefiles-1\") pod \"35556def-a4de-4ccf-984f-7700074c610c\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " Jul 08 22:16:54.350301 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.350220 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-configmap-metrics-client-ca\") pod \"35556def-a4de-4ccf-984f-7700074c610c\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " Jul 08 22:16:54.350301 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.350246 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/35556def-a4de-4ccf-984f-7700074c610c-tls-assets\") pod \"35556def-a4de-4ccf-984f-7700074c610c\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " Jul 08 22:16:54.350301 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.350279 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-web-config\") pod \"35556def-a4de-4ccf-984f-7700074c610c\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " Jul 08 22:16:54.350749 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.350321 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-secret-prometheus-k8s-tls\") pod \"35556def-a4de-4ccf-984f-7700074c610c\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " Jul 08 22:16:54.350749 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.350350 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-configmap-kubelet-serving-ca-bundle\") pod \"35556def-a4de-4ccf-984f-7700074c610c\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " Jul 08 22:16:54.350749 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.350378 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-prometheus-k8s-rulefiles-0\") pod \"35556def-a4de-4ccf-984f-7700074c610c\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " Jul 08 22:16:54.350749 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.350415 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-prometheus-trusted-ca-bundle\") pod \"35556def-a4de-4ccf-984f-7700074c610c\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " Jul 08 22:16:54.350749 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.350442 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-secret-kube-rbac-proxy\") pod \"35556def-a4de-4ccf-984f-7700074c610c\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " Jul 08 22:16:54.350749 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.350445 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "35556def-a4de-4ccf-984f-7700074c610c" (UID: "35556def-a4de-4ccf-984f-7700074c610c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 22:16:54.350749 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.350478 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-configmap-serving-certs-ca-bundle\") pod \"35556def-a4de-4ccf-984f-7700074c610c\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " Jul 08 22:16:54.350749 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.350503 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/35556def-a4de-4ccf-984f-7700074c610c-prometheus-k8s-db\") pod \"35556def-a4de-4ccf-984f-7700074c610c\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " Jul 08 22:16:54.350749 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.350538 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"35556def-a4de-4ccf-984f-7700074c610c\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " Jul 08 22:16:54.350749 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.350573 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-thanos-prometheus-http-client-file\") pod \"35556def-a4de-4ccf-984f-7700074c610c\" (UID: \"35556def-a4de-4ccf-984f-7700074c610c\") " Jul 08 22:16:54.350749 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.350625 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "35556def-a4de-4ccf-984f-7700074c610c" (UID: "35556def-a4de-4ccf-984f-7700074c610c"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 22:16:54.351298 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.350821 2563 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-configmap-metrics-client-ca\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:54.351298 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.350843 2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:54.351298 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.351148 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "35556def-a4de-4ccf-984f-7700074c610c" (UID: "35556def-a4de-4ccf-984f-7700074c610c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 22:16:54.352756 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.351973 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "35556def-a4de-4ccf-984f-7700074c610c" (UID: "35556def-a4de-4ccf-984f-7700074c610c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 22:16:54.353480 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.353260 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "35556def-a4de-4ccf-984f-7700074c610c" (UID: "35556def-a4de-4ccf-984f-7700074c610c"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 22:16:54.353480 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.353448 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "35556def-a4de-4ccf-984f-7700074c610c" (UID: "35556def-a4de-4ccf-984f-7700074c610c"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 22:16:54.353835 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.353697 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35556def-a4de-4ccf-984f-7700074c610c-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "35556def-a4de-4ccf-984f-7700074c610c" (UID: "35556def-a4de-4ccf-984f-7700074c610c"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:16:54.353835 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.353805 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "35556def-a4de-4ccf-984f-7700074c610c" (UID: "35556def-a4de-4ccf-984f-7700074c610c"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 22:16:54.354189 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.354152 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35556def-a4de-4ccf-984f-7700074c610c-config-out" (OuterVolumeSpecName: "config-out") pod "35556def-a4de-4ccf-984f-7700074c610c" (UID: "35556def-a4de-4ccf-984f-7700074c610c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:16:54.354468 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.354439 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "35556def-a4de-4ccf-984f-7700074c610c" (UID: "35556def-a4de-4ccf-984f-7700074c610c"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 22:16:54.354596 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.354574 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "35556def-a4de-4ccf-984f-7700074c610c" (UID: "35556def-a4de-4ccf-984f-7700074c610c"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 22:16:54.354835 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.354812 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "35556def-a4de-4ccf-984f-7700074c610c" (UID: "35556def-a4de-4ccf-984f-7700074c610c"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 22:16:54.354977 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.354953 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35556def-a4de-4ccf-984f-7700074c610c-kube-api-access-m6njg" (OuterVolumeSpecName: "kube-api-access-m6njg") pod "35556def-a4de-4ccf-984f-7700074c610c" (UID: "35556def-a4de-4ccf-984f-7700074c610c"). InnerVolumeSpecName "kube-api-access-m6njg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 22:16:54.355283 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.355255 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-config" (OuterVolumeSpecName: "config") pod "35556def-a4de-4ccf-984f-7700074c610c" (UID: "35556def-a4de-4ccf-984f-7700074c610c"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 22:16:54.355395 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.355376 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "35556def-a4de-4ccf-984f-7700074c610c" (UID: "35556def-a4de-4ccf-984f-7700074c610c"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 22:16:54.355689 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.355666 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "35556def-a4de-4ccf-984f-7700074c610c" (UID: "35556def-a4de-4ccf-984f-7700074c610c"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 22:16:54.355767 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.355747 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "35556def-a4de-4ccf-984f-7700074c610c" (UID: "35556def-a4de-4ccf-984f-7700074c610c"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 22:16:54.356165 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.356140 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35556def-a4de-4ccf-984f-7700074c610c-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "35556def-a4de-4ccf-984f-7700074c610c" (UID: "35556def-a4de-4ccf-984f-7700074c610c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 22:16:54.356248 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.356206 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "35556def-a4de-4ccf-984f-7700074c610c" (UID: "35556def-a4de-4ccf-984f-7700074c610c"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 22:16:54.365315 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.365283 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-web-config" (OuterVolumeSpecName: "web-config") pod "35556def-a4de-4ccf-984f-7700074c610c" (UID: "35556def-a4de-4ccf-984f-7700074c610c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 22:16:54.451827 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.451789 2563 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/35556def-a4de-4ccf-984f-7700074c610c-tls-assets\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:54.451827 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.451822 2563 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-web-config\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:54.451827 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.451833 2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-secret-prometheus-k8s-tls\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:54.452088 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.451843 2563 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:54.452088 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.451853 2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:54.452088 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.451863 2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-prometheus-trusted-ca-bundle\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:54.452088 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.451871 2563 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-secret-kube-rbac-proxy\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:54.452088 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.451879 2563 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:54.452088 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.451910 2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/35556def-a4de-4ccf-984f-7700074c610c-prometheus-k8s-db\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:54.452088 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.451919 2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:54.452088 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.451928 2563 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-thanos-prometheus-http-client-file\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:54.452088 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.451936 2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:54.452088 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.451945 2563 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-secret-metrics-client-certs\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:54.452088 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.451955 2563 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/35556def-a4de-4ccf-984f-7700074c610c-config-out\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:54.452088 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.451963 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6njg\" (UniqueName: \"kubernetes.io/projected/35556def-a4de-4ccf-984f-7700074c610c-kube-api-access-m6njg\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:54.452088 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.451971 2563 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-secret-grpc-tls\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:54.452088 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.451978 2563 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/35556def-a4de-4ccf-984f-7700074c610c-config\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:54.452088 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:54.451986 2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/35556def-a4de-4ccf-984f-7700074c610c-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:16:55.115068 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.115025 2563 generic.go:356] "Generic (PLEG): container finished" podID="35556def-a4de-4ccf-984f-7700074c610c" containerID="9e81381c60d659d4968272830a489c51bd947a116e5c7944656bfe9a81c3bfa3" exitCode=0 Jul 08 22:16:55.115068 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.115050 2563 generic.go:356] "Generic (PLEG): container finished" podID="35556def-a4de-4ccf-984f-7700074c610c" containerID="31f993706c163fefd47d617879ea96a1d5abe225aea9397c013ffdb0091d0f5d" exitCode=0 Jul 08 22:16:55.115068 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.115058 2563 generic.go:356] "Generic (PLEG): container finished" podID="35556def-a4de-4ccf-984f-7700074c610c" containerID="4bfba3ee8d60dc9314129bd39e1b86cbcbc83ebf236b564445829d927908d108" exitCode=0 Jul 08 22:16:55.115068 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.115069 2563 generic.go:356] "Generic (PLEG): container finished" podID="35556def-a4de-4ccf-984f-7700074c610c" containerID="ffe5f3b2d2d8471db7d401688f7a255314d15513f6e2cc336f433bda3baba389" exitCode=0 Jul 08 22:16:55.115068 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.115076 2563 generic.go:356] "Generic (PLEG): container finished" podID="35556def-a4de-4ccf-984f-7700074c610c" containerID="f02f34d86a139d7519b22fea82b13f95dd949c23fdcb75fd8136b1fdce54f7f4" exitCode=0 Jul 08 22:16:55.115801 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.115084 2563 generic.go:356] "Generic (PLEG): container finished" podID="35556def-a4de-4ccf-984f-7700074c610c" containerID="2a017ef95710dc8de2a3946e157b9a07080bc59c5ad5f02d1005bb710b45d7b8" exitCode=0 Jul 08 22:16:55.115801 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.115110 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.115801 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.115105 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"35556def-a4de-4ccf-984f-7700074c610c","Type":"ContainerDied","Data":"9e81381c60d659d4968272830a489c51bd947a116e5c7944656bfe9a81c3bfa3"} Jul 08 22:16:55.115801 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.115270 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"35556def-a4de-4ccf-984f-7700074c610c","Type":"ContainerDied","Data":"31f993706c163fefd47d617879ea96a1d5abe225aea9397c013ffdb0091d0f5d"} Jul 08 22:16:55.115801 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.115298 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"35556def-a4de-4ccf-984f-7700074c610c","Type":"ContainerDied","Data":"4bfba3ee8d60dc9314129bd39e1b86cbcbc83ebf236b564445829d927908d108"} Jul 08 22:16:55.115801 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.115323 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"35556def-a4de-4ccf-984f-7700074c610c","Type":"ContainerDied","Data":"ffe5f3b2d2d8471db7d401688f7a255314d15513f6e2cc336f433bda3baba389"} Jul 08 22:16:55.115801 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.115341 2563 scope.go:117] "RemoveContainer" containerID="9e81381c60d659d4968272830a489c51bd947a116e5c7944656bfe9a81c3bfa3" Jul 08 22:16:55.115801 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.115347 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"35556def-a4de-4ccf-984f-7700074c610c","Type":"ContainerDied","Data":"f02f34d86a139d7519b22fea82b13f95dd949c23fdcb75fd8136b1fdce54f7f4"} Jul 08 22:16:55.115801 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.115372 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"35556def-a4de-4ccf-984f-7700074c610c","Type":"ContainerDied","Data":"2a017ef95710dc8de2a3946e157b9a07080bc59c5ad5f02d1005bb710b45d7b8"} Jul 08 22:16:55.115801 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.115395 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"35556def-a4de-4ccf-984f-7700074c610c","Type":"ContainerDied","Data":"836274ef3990764502bf3c4f96be4073ea482659b53a637a692b84f733436a45"} Jul 08 22:16:55.123684 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.123670 2563 scope.go:117] "RemoveContainer" containerID="31f993706c163fefd47d617879ea96a1d5abe225aea9397c013ffdb0091d0f5d" Jul 08 22:16:55.150544 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.150520 2563 scope.go:117] "RemoveContainer" containerID="4bfba3ee8d60dc9314129bd39e1b86cbcbc83ebf236b564445829d927908d108" Jul 08 22:16:55.158188 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.158165 2563 scope.go:117] "RemoveContainer" containerID="ffe5f3b2d2d8471db7d401688f7a255314d15513f6e2cc336f433bda3baba389" Jul 08 22:16:55.160032 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.160008 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 22:16:55.163920 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.163865 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 22:16:55.166399 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.166381 2563 scope.go:117] "RemoveContainer" containerID="f02f34d86a139d7519b22fea82b13f95dd949c23fdcb75fd8136b1fdce54f7f4" Jul 08 22:16:55.175398 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.175375 2563 scope.go:117] "RemoveContainer" containerID="2a017ef95710dc8de2a3946e157b9a07080bc59c5ad5f02d1005bb710b45d7b8" Jul 08 22:16:55.183355 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.183333 2563 scope.go:117] "RemoveContainer" containerID="ea076cfbd8e0d0031cc9ff72c3fb76fcb4efaa03bfa0ae37ab614269507509a1" Jul 08 22:16:55.190630 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.190596 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 22:16:55.191023 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.191004 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35556def-a4de-4ccf-984f-7700074c610c" containerName="init-config-reloader" Jul 08 22:16:55.191072 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.191027 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="35556def-a4de-4ccf-984f-7700074c610c" containerName="init-config-reloader" Jul 08 22:16:55.191072 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.191039 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35556def-a4de-4ccf-984f-7700074c610c" containerName="thanos-sidecar" Jul 08 22:16:55.191072 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.191049 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="35556def-a4de-4ccf-984f-7700074c610c" containerName="thanos-sidecar" Jul 08 22:16:55.191072 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.191061 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35556def-a4de-4ccf-984f-7700074c610c" containerName="kube-rbac-proxy" Jul 08 22:16:55.191206 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.191070 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="35556def-a4de-4ccf-984f-7700074c610c" containerName="kube-rbac-proxy" Jul 08 22:16:55.191206 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.191117 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28e04205-7edb-4465-928d-ef26a27ae65c" containerName="console" Jul 08 22:16:55.191206 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.191125 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="28e04205-7edb-4465-928d-ef26a27ae65c" containerName="console" Jul 08 22:16:55.191206 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.191141 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35556def-a4de-4ccf-984f-7700074c610c" containerName="prometheus" Jul 08 22:16:55.191206 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.191150 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="35556def-a4de-4ccf-984f-7700074c610c" containerName="prometheus" Jul 08 22:16:55.191206 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.191162 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35556def-a4de-4ccf-984f-7700074c610c" containerName="kube-rbac-proxy-web" Jul 08 22:16:55.191206 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.191171 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="35556def-a4de-4ccf-984f-7700074c610c" containerName="kube-rbac-proxy-web" Jul 08 22:16:55.191206 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.191183 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35556def-a4de-4ccf-984f-7700074c610c" containerName="kube-rbac-proxy-thanos" Jul 08 22:16:55.191206 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.191191 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="35556def-a4de-4ccf-984f-7700074c610c" containerName="kube-rbac-proxy-thanos" Jul 08 22:16:55.191482 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.191210 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d62c766-320a-4fea-ac5a-23db8702e521" containerName="console" Jul 08 22:16:55.191482 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.191219 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d62c766-320a-4fea-ac5a-23db8702e521" containerName="console" Jul 08 22:16:55.191482 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.191237 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35556def-a4de-4ccf-984f-7700074c610c" containerName="config-reloader" Jul 08 22:16:55.191482 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.191246 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="35556def-a4de-4ccf-984f-7700074c610c" containerName="config-reloader" Jul 08 22:16:55.191482 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.191257 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c022be9-ee23-4a42-b174-f502add283f7" containerName="console" Jul 08 22:16:55.191482 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.191265 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c022be9-ee23-4a42-b174-f502add283f7" containerName="console" Jul 08 22:16:55.191482 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.191331 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="35556def-a4de-4ccf-984f-7700074c610c" containerName="prometheus" Jul 08 22:16:55.191482 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.191342 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="35556def-a4de-4ccf-984f-7700074c610c" containerName="kube-rbac-proxy-thanos" Jul 08 22:16:55.191482 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.191352 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="35556def-a4de-4ccf-984f-7700074c610c" containerName="thanos-sidecar" Jul 08 22:16:55.191482 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.191360 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d62c766-320a-4fea-ac5a-23db8702e521" containerName="console" Jul 08 22:16:55.191482 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.191370 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c022be9-ee23-4a42-b174-f502add283f7" containerName="console" Jul 08 22:16:55.191482 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.191378 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="35556def-a4de-4ccf-984f-7700074c610c" containerName="config-reloader" Jul 08 22:16:55.191482 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.191387 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="35556def-a4de-4ccf-984f-7700074c610c" containerName="kube-rbac-proxy-web" Jul 08 22:16:55.191482 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.191394 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="35556def-a4de-4ccf-984f-7700074c610c" containerName="kube-rbac-proxy" Jul 08 22:16:55.191482 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.191402 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="28e04205-7edb-4465-928d-ef26a27ae65c" containerName="console" Jul 08 22:16:55.192209 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.192188 2563 scope.go:117] "RemoveContainer" containerID="9e81381c60d659d4968272830a489c51bd947a116e5c7944656bfe9a81c3bfa3" Jul 08 22:16:55.192563 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:16:55.192535 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e81381c60d659d4968272830a489c51bd947a116e5c7944656bfe9a81c3bfa3\": container with ID starting with 9e81381c60d659d4968272830a489c51bd947a116e5c7944656bfe9a81c3bfa3 not found: ID does not exist" containerID="9e81381c60d659d4968272830a489c51bd947a116e5c7944656bfe9a81c3bfa3" Jul 08 22:16:55.192639 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.192573 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e81381c60d659d4968272830a489c51bd947a116e5c7944656bfe9a81c3bfa3"} err="failed to get container status \"9e81381c60d659d4968272830a489c51bd947a116e5c7944656bfe9a81c3bfa3\": rpc error: code = NotFound desc = could not find container \"9e81381c60d659d4968272830a489c51bd947a116e5c7944656bfe9a81c3bfa3\": container with ID starting with 9e81381c60d659d4968272830a489c51bd947a116e5c7944656bfe9a81c3bfa3 not found: ID does not exist" Jul 08 22:16:55.192639 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.192593 2563 scope.go:117] "RemoveContainer" containerID="31f993706c163fefd47d617879ea96a1d5abe225aea9397c013ffdb0091d0f5d" Jul 08 22:16:55.192911 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:16:55.192873 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31f993706c163fefd47d617879ea96a1d5abe225aea9397c013ffdb0091d0f5d\": container with ID starting with 31f993706c163fefd47d617879ea96a1d5abe225aea9397c013ffdb0091d0f5d not found: ID does not exist" containerID="31f993706c163fefd47d617879ea96a1d5abe225aea9397c013ffdb0091d0f5d" Jul 08 22:16:55.193010 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.192917 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31f993706c163fefd47d617879ea96a1d5abe225aea9397c013ffdb0091d0f5d"} err="failed to get container status \"31f993706c163fefd47d617879ea96a1d5abe225aea9397c013ffdb0091d0f5d\": rpc error: code = NotFound desc = could not find container \"31f993706c163fefd47d617879ea96a1d5abe225aea9397c013ffdb0091d0f5d\": container with ID starting with 31f993706c163fefd47d617879ea96a1d5abe225aea9397c013ffdb0091d0f5d not found: ID does not exist" Jul 08 22:16:55.193010 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.192938 2563 scope.go:117] "RemoveContainer" containerID="4bfba3ee8d60dc9314129bd39e1b86cbcbc83ebf236b564445829d927908d108" Jul 08 22:16:55.193248 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:16:55.193225 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bfba3ee8d60dc9314129bd39e1b86cbcbc83ebf236b564445829d927908d108\": container with ID starting with 4bfba3ee8d60dc9314129bd39e1b86cbcbc83ebf236b564445829d927908d108 not found: ID does not exist" containerID="4bfba3ee8d60dc9314129bd39e1b86cbcbc83ebf236b564445829d927908d108" Jul 08 22:16:55.193294 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.193255 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bfba3ee8d60dc9314129bd39e1b86cbcbc83ebf236b564445829d927908d108"} err="failed to get container status \"4bfba3ee8d60dc9314129bd39e1b86cbcbc83ebf236b564445829d927908d108\": rpc error: code = NotFound desc = could not find container \"4bfba3ee8d60dc9314129bd39e1b86cbcbc83ebf236b564445829d927908d108\": container with ID starting with 4bfba3ee8d60dc9314129bd39e1b86cbcbc83ebf236b564445829d927908d108 not found: ID does not exist" Jul 08 22:16:55.193294 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.193276 2563 scope.go:117] "RemoveContainer" containerID="ffe5f3b2d2d8471db7d401688f7a255314d15513f6e2cc336f433bda3baba389" Jul 08 22:16:55.193575 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:16:55.193554 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffe5f3b2d2d8471db7d401688f7a255314d15513f6e2cc336f433bda3baba389\": container with ID starting with ffe5f3b2d2d8471db7d401688f7a255314d15513f6e2cc336f433bda3baba389 not found: ID does not exist" containerID="ffe5f3b2d2d8471db7d401688f7a255314d15513f6e2cc336f433bda3baba389" Jul 08 22:16:55.193632 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.193580 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffe5f3b2d2d8471db7d401688f7a255314d15513f6e2cc336f433bda3baba389"} err="failed to get container status \"ffe5f3b2d2d8471db7d401688f7a255314d15513f6e2cc336f433bda3baba389\": rpc error: code = NotFound desc = could not find container \"ffe5f3b2d2d8471db7d401688f7a255314d15513f6e2cc336f433bda3baba389\": container with ID starting with ffe5f3b2d2d8471db7d401688f7a255314d15513f6e2cc336f433bda3baba389 not found: ID does not exist" Jul 08 22:16:55.193632 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.193597 2563 scope.go:117] "RemoveContainer" containerID="f02f34d86a139d7519b22fea82b13f95dd949c23fdcb75fd8136b1fdce54f7f4" Jul 08 22:16:55.193925 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:16:55.193896 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f02f34d86a139d7519b22fea82b13f95dd949c23fdcb75fd8136b1fdce54f7f4\": container with ID starting with f02f34d86a139d7519b22fea82b13f95dd949c23fdcb75fd8136b1fdce54f7f4 not found: ID does not exist" containerID="f02f34d86a139d7519b22fea82b13f95dd949c23fdcb75fd8136b1fdce54f7f4" Jul 08 22:16:55.194039 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.193927 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f02f34d86a139d7519b22fea82b13f95dd949c23fdcb75fd8136b1fdce54f7f4"} err="failed to get container status \"f02f34d86a139d7519b22fea82b13f95dd949c23fdcb75fd8136b1fdce54f7f4\": rpc error: code = NotFound desc = could not find container \"f02f34d86a139d7519b22fea82b13f95dd949c23fdcb75fd8136b1fdce54f7f4\": container with ID starting with f02f34d86a139d7519b22fea82b13f95dd949c23fdcb75fd8136b1fdce54f7f4 not found: ID does not exist" Jul 08 22:16:55.194039 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.193952 2563 scope.go:117] "RemoveContainer" containerID="2a017ef95710dc8de2a3946e157b9a07080bc59c5ad5f02d1005bb710b45d7b8" Jul 08 22:16:55.194228 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:16:55.194211 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a017ef95710dc8de2a3946e157b9a07080bc59c5ad5f02d1005bb710b45d7b8\": container with ID starting with 2a017ef95710dc8de2a3946e157b9a07080bc59c5ad5f02d1005bb710b45d7b8 not found: ID does not exist" containerID="2a017ef95710dc8de2a3946e157b9a07080bc59c5ad5f02d1005bb710b45d7b8" Jul 08 22:16:55.194291 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.194232 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a017ef95710dc8de2a3946e157b9a07080bc59c5ad5f02d1005bb710b45d7b8"} err="failed to get container status \"2a017ef95710dc8de2a3946e157b9a07080bc59c5ad5f02d1005bb710b45d7b8\": rpc error: code = NotFound desc = could not find container \"2a017ef95710dc8de2a3946e157b9a07080bc59c5ad5f02d1005bb710b45d7b8\": container with ID starting with 2a017ef95710dc8de2a3946e157b9a07080bc59c5ad5f02d1005bb710b45d7b8 not found: ID does not exist" Jul 08 22:16:55.194291 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.194246 2563 scope.go:117] "RemoveContainer" containerID="ea076cfbd8e0d0031cc9ff72c3fb76fcb4efaa03bfa0ae37ab614269507509a1" Jul 08 22:16:55.194481 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:16:55.194462 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea076cfbd8e0d0031cc9ff72c3fb76fcb4efaa03bfa0ae37ab614269507509a1\": container with ID starting with ea076cfbd8e0d0031cc9ff72c3fb76fcb4efaa03bfa0ae37ab614269507509a1 not found: ID does not exist" containerID="ea076cfbd8e0d0031cc9ff72c3fb76fcb4efaa03bfa0ae37ab614269507509a1" Jul 08 22:16:55.194556 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.194485 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea076cfbd8e0d0031cc9ff72c3fb76fcb4efaa03bfa0ae37ab614269507509a1"} err="failed to get container status \"ea076cfbd8e0d0031cc9ff72c3fb76fcb4efaa03bfa0ae37ab614269507509a1\": rpc error: code = NotFound desc = could not find container \"ea076cfbd8e0d0031cc9ff72c3fb76fcb4efaa03bfa0ae37ab614269507509a1\": container with ID starting with ea076cfbd8e0d0031cc9ff72c3fb76fcb4efaa03bfa0ae37ab614269507509a1 not found: ID does not exist" Jul 08 22:16:55.194556 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.194500 2563 scope.go:117] "RemoveContainer" containerID="9e81381c60d659d4968272830a489c51bd947a116e5c7944656bfe9a81c3bfa3" Jul 08 22:16:55.194708 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.194687 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e81381c60d659d4968272830a489c51bd947a116e5c7944656bfe9a81c3bfa3"} err="failed to get container status \"9e81381c60d659d4968272830a489c51bd947a116e5c7944656bfe9a81c3bfa3\": rpc error: code = NotFound desc = could not find container \"9e81381c60d659d4968272830a489c51bd947a116e5c7944656bfe9a81c3bfa3\": container with ID starting with 9e81381c60d659d4968272830a489c51bd947a116e5c7944656bfe9a81c3bfa3 not found: ID does not exist" Jul 08 22:16:55.194708 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.194706 2563 scope.go:117] "RemoveContainer" containerID="31f993706c163fefd47d617879ea96a1d5abe225aea9397c013ffdb0091d0f5d" Jul 08 22:16:55.194953 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.194934 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31f993706c163fefd47d617879ea96a1d5abe225aea9397c013ffdb0091d0f5d"} err="failed to get container status \"31f993706c163fefd47d617879ea96a1d5abe225aea9397c013ffdb0091d0f5d\": rpc error: code = NotFound desc = could not find container \"31f993706c163fefd47d617879ea96a1d5abe225aea9397c013ffdb0091d0f5d\": container with ID starting with 31f993706c163fefd47d617879ea96a1d5abe225aea9397c013ffdb0091d0f5d not found: ID does not exist" Jul 08 22:16:55.194953 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.194953 2563 scope.go:117] "RemoveContainer" containerID="4bfba3ee8d60dc9314129bd39e1b86cbcbc83ebf236b564445829d927908d108" Jul 08 22:16:55.195205 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.195184 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bfba3ee8d60dc9314129bd39e1b86cbcbc83ebf236b564445829d927908d108"} err="failed to get container status \"4bfba3ee8d60dc9314129bd39e1b86cbcbc83ebf236b564445829d927908d108\": rpc error: code = NotFound desc = could not find container \"4bfba3ee8d60dc9314129bd39e1b86cbcbc83ebf236b564445829d927908d108\": container with ID starting with 4bfba3ee8d60dc9314129bd39e1b86cbcbc83ebf236b564445829d927908d108 not found: ID does not exist" Jul 08 22:16:55.195283 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.195207 2563 scope.go:117] "RemoveContainer" containerID="ffe5f3b2d2d8471db7d401688f7a255314d15513f6e2cc336f433bda3baba389" Jul 08 22:16:55.195452 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.195433 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffe5f3b2d2d8471db7d401688f7a255314d15513f6e2cc336f433bda3baba389"} err="failed to get container status \"ffe5f3b2d2d8471db7d401688f7a255314d15513f6e2cc336f433bda3baba389\": rpc error: code = NotFound desc = could not find container \"ffe5f3b2d2d8471db7d401688f7a255314d15513f6e2cc336f433bda3baba389\": container with ID starting with ffe5f3b2d2d8471db7d401688f7a255314d15513f6e2cc336f433bda3baba389 not found: ID does not exist" Jul 08 22:16:55.195493 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.195454 2563 scope.go:117] "RemoveContainer" containerID="f02f34d86a139d7519b22fea82b13f95dd949c23fdcb75fd8136b1fdce54f7f4" Jul 08 22:16:55.195670 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.195656 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f02f34d86a139d7519b22fea82b13f95dd949c23fdcb75fd8136b1fdce54f7f4"} err="failed to get container status \"f02f34d86a139d7519b22fea82b13f95dd949c23fdcb75fd8136b1fdce54f7f4\": rpc error: code = NotFound desc = could not find container \"f02f34d86a139d7519b22fea82b13f95dd949c23fdcb75fd8136b1fdce54f7f4\": container with ID starting with f02f34d86a139d7519b22fea82b13f95dd949c23fdcb75fd8136b1fdce54f7f4 not found: ID does not exist" Jul 08 22:16:55.195670 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.195670 2563 scope.go:117] "RemoveContainer" containerID="2a017ef95710dc8de2a3946e157b9a07080bc59c5ad5f02d1005bb710b45d7b8" Jul 08 22:16:55.195773 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.195757 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.195874 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.195856 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a017ef95710dc8de2a3946e157b9a07080bc59c5ad5f02d1005bb710b45d7b8"} err="failed to get container status \"2a017ef95710dc8de2a3946e157b9a07080bc59c5ad5f02d1005bb710b45d7b8\": rpc error: code = NotFound desc = could not find container \"2a017ef95710dc8de2a3946e157b9a07080bc59c5ad5f02d1005bb710b45d7b8\": container with ID starting with 2a017ef95710dc8de2a3946e157b9a07080bc59c5ad5f02d1005bb710b45d7b8 not found: ID does not exist" Jul 08 22:16:55.195951 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.195876 2563 scope.go:117] "RemoveContainer" containerID="ea076cfbd8e0d0031cc9ff72c3fb76fcb4efaa03bfa0ae37ab614269507509a1" Jul 08 22:16:55.196399 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.196368 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea076cfbd8e0d0031cc9ff72c3fb76fcb4efaa03bfa0ae37ab614269507509a1"} err="failed to get container status \"ea076cfbd8e0d0031cc9ff72c3fb76fcb4efaa03bfa0ae37ab614269507509a1\": rpc error: code = NotFound desc = could not find container \"ea076cfbd8e0d0031cc9ff72c3fb76fcb4efaa03bfa0ae37ab614269507509a1\": container with ID starting with ea076cfbd8e0d0031cc9ff72c3fb76fcb4efaa03bfa0ae37ab614269507509a1 not found: ID does not exist" Jul 08 22:16:55.196493 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.196401 2563 scope.go:117] "RemoveContainer" containerID="9e81381c60d659d4968272830a489c51bd947a116e5c7944656bfe9a81c3bfa3" Jul 08 22:16:55.196671 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.196645 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e81381c60d659d4968272830a489c51bd947a116e5c7944656bfe9a81c3bfa3"} err="failed to get container status \"9e81381c60d659d4968272830a489c51bd947a116e5c7944656bfe9a81c3bfa3\": rpc error: code = NotFound desc = could not find container \"9e81381c60d659d4968272830a489c51bd947a116e5c7944656bfe9a81c3bfa3\": container with ID starting with 9e81381c60d659d4968272830a489c51bd947a116e5c7944656bfe9a81c3bfa3 not found: ID does not exist" Jul 08 22:16:55.196743 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.196672 2563 scope.go:117] "RemoveContainer" containerID="31f993706c163fefd47d617879ea96a1d5abe225aea9397c013ffdb0091d0f5d" Jul 08 22:16:55.196914 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.196873 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31f993706c163fefd47d617879ea96a1d5abe225aea9397c013ffdb0091d0f5d"} err="failed to get container status \"31f993706c163fefd47d617879ea96a1d5abe225aea9397c013ffdb0091d0f5d\": rpc error: code = NotFound desc = could not find container \"31f993706c163fefd47d617879ea96a1d5abe225aea9397c013ffdb0091d0f5d\": container with ID starting with 31f993706c163fefd47d617879ea96a1d5abe225aea9397c013ffdb0091d0f5d not found: ID does not exist" Jul 08 22:16:55.196990 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.196914 2563 scope.go:117] "RemoveContainer" containerID="4bfba3ee8d60dc9314129bd39e1b86cbcbc83ebf236b564445829d927908d108" Jul 08 22:16:55.197160 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.197134 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bfba3ee8d60dc9314129bd39e1b86cbcbc83ebf236b564445829d927908d108"} err="failed to get container status \"4bfba3ee8d60dc9314129bd39e1b86cbcbc83ebf236b564445829d927908d108\": rpc error: code = NotFound desc = could not find container \"4bfba3ee8d60dc9314129bd39e1b86cbcbc83ebf236b564445829d927908d108\": container with ID starting with 4bfba3ee8d60dc9314129bd39e1b86cbcbc83ebf236b564445829d927908d108 not found: ID does not exist" Jul 08 22:16:55.197244 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.197160 2563 scope.go:117] "RemoveContainer" containerID="ffe5f3b2d2d8471db7d401688f7a255314d15513f6e2cc336f433bda3baba389" Jul 08 22:16:55.197387 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.197367 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffe5f3b2d2d8471db7d401688f7a255314d15513f6e2cc336f433bda3baba389"} err="failed to get container status \"ffe5f3b2d2d8471db7d401688f7a255314d15513f6e2cc336f433bda3baba389\": rpc error: code = NotFound desc = could not find container \"ffe5f3b2d2d8471db7d401688f7a255314d15513f6e2cc336f433bda3baba389\": container with ID starting with ffe5f3b2d2d8471db7d401688f7a255314d15513f6e2cc336f433bda3baba389 not found: ID does not exist" Jul 08 22:16:55.197431 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.197388 2563 scope.go:117] "RemoveContainer" containerID="f02f34d86a139d7519b22fea82b13f95dd949c23fdcb75fd8136b1fdce54f7f4" Jul 08 22:16:55.197579 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.197562 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f02f34d86a139d7519b22fea82b13f95dd949c23fdcb75fd8136b1fdce54f7f4"} err="failed to get container status \"f02f34d86a139d7519b22fea82b13f95dd949c23fdcb75fd8136b1fdce54f7f4\": rpc error: code = NotFound desc = could not find container \"f02f34d86a139d7519b22fea82b13f95dd949c23fdcb75fd8136b1fdce54f7f4\": container with ID starting with f02f34d86a139d7519b22fea82b13f95dd949c23fdcb75fd8136b1fdce54f7f4 not found: ID does not exist" Jul 08 22:16:55.197620 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.197579 2563 scope.go:117] "RemoveContainer" containerID="2a017ef95710dc8de2a3946e157b9a07080bc59c5ad5f02d1005bb710b45d7b8" Jul 08 22:16:55.197757 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.197742 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a017ef95710dc8de2a3946e157b9a07080bc59c5ad5f02d1005bb710b45d7b8"} err="failed to get container status \"2a017ef95710dc8de2a3946e157b9a07080bc59c5ad5f02d1005bb710b45d7b8\": rpc error: code = NotFound desc = could not find container \"2a017ef95710dc8de2a3946e157b9a07080bc59c5ad5f02d1005bb710b45d7b8\": container with ID starting with 2a017ef95710dc8de2a3946e157b9a07080bc59c5ad5f02d1005bb710b45d7b8 not found: ID does not exist" Jul 08 22:16:55.197794 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.197757 2563 scope.go:117] "RemoveContainer" containerID="ea076cfbd8e0d0031cc9ff72c3fb76fcb4efaa03bfa0ae37ab614269507509a1" Jul 08 22:16:55.197991 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.197938 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea076cfbd8e0d0031cc9ff72c3fb76fcb4efaa03bfa0ae37ab614269507509a1"} err="failed to get container status \"ea076cfbd8e0d0031cc9ff72c3fb76fcb4efaa03bfa0ae37ab614269507509a1\": rpc error: code = NotFound desc = could not find container \"ea076cfbd8e0d0031cc9ff72c3fb76fcb4efaa03bfa0ae37ab614269507509a1\": container with ID starting with ea076cfbd8e0d0031cc9ff72c3fb76fcb4efaa03bfa0ae37ab614269507509a1 not found: ID does not exist" Jul 08 22:16:55.198047 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.197993 2563 scope.go:117] "RemoveContainer" containerID="9e81381c60d659d4968272830a489c51bd947a116e5c7944656bfe9a81c3bfa3" Jul 08 22:16:55.198297 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.198271 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e81381c60d659d4968272830a489c51bd947a116e5c7944656bfe9a81c3bfa3"} err="failed to get container status \"9e81381c60d659d4968272830a489c51bd947a116e5c7944656bfe9a81c3bfa3\": rpc error: code = NotFound desc = could not find container \"9e81381c60d659d4968272830a489c51bd947a116e5c7944656bfe9a81c3bfa3\": container with ID starting with 9e81381c60d659d4968272830a489c51bd947a116e5c7944656bfe9a81c3bfa3 not found: ID does not exist" Jul 08 22:16:55.198403 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.198298 2563 scope.go:117] "RemoveContainer" containerID="31f993706c163fefd47d617879ea96a1d5abe225aea9397c013ffdb0091d0f5d" Jul 08 22:16:55.198596 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.198578 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 08 22:16:55.198596 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.198581 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 08 22:16:55.198732 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.198581 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4t4b56c4nce9k\"" Jul 08 22:16:55.198732 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.198581 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 08 22:16:55.198825 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.198792 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31f993706c163fefd47d617879ea96a1d5abe225aea9397c013ffdb0091d0f5d"} err="failed to get container status \"31f993706c163fefd47d617879ea96a1d5abe225aea9397c013ffdb0091d0f5d\": rpc error: code = NotFound desc = could not find container \"31f993706c163fefd47d617879ea96a1d5abe225aea9397c013ffdb0091d0f5d\": container with ID starting with 31f993706c163fefd47d617879ea96a1d5abe225aea9397c013ffdb0091d0f5d not found: ID does not exist" Jul 08 22:16:55.198825 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.198815 2563 scope.go:117] "RemoveContainer" containerID="4bfba3ee8d60dc9314129bd39e1b86cbcbc83ebf236b564445829d927908d108" Jul 08 22:16:55.199137 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.199109 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bfba3ee8d60dc9314129bd39e1b86cbcbc83ebf236b564445829d927908d108"} err="failed to get container status \"4bfba3ee8d60dc9314129bd39e1b86cbcbc83ebf236b564445829d927908d108\": rpc error: code = NotFound desc = could not find container \"4bfba3ee8d60dc9314129bd39e1b86cbcbc83ebf236b564445829d927908d108\": container with ID starting with 4bfba3ee8d60dc9314129bd39e1b86cbcbc83ebf236b564445829d927908d108 not found: ID does not exist" Jul 08 22:16:55.199137 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.199137 2563 scope.go:117] "RemoveContainer" containerID="ffe5f3b2d2d8471db7d401688f7a255314d15513f6e2cc336f433bda3baba389" Jul 08 22:16:55.199321 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.199156 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 08 22:16:55.199321 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.199187 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 08 22:16:55.199321 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.199301 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 08 22:16:55.199618 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.199404 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 08 22:16:55.199618 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.199411 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 08 22:16:55.199618 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.199402 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffe5f3b2d2d8471db7d401688f7a255314d15513f6e2cc336f433bda3baba389"} err="failed to get container status \"ffe5f3b2d2d8471db7d401688f7a255314d15513f6e2cc336f433bda3baba389\": rpc error: code = NotFound desc = could not find container \"ffe5f3b2d2d8471db7d401688f7a255314d15513f6e2cc336f433bda3baba389\": container with ID starting with ffe5f3b2d2d8471db7d401688f7a255314d15513f6e2cc336f433bda3baba389 not found: ID does not exist" Jul 08 22:16:55.199618 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.199436 2563 scope.go:117] "RemoveContainer" containerID="f02f34d86a139d7519b22fea82b13f95dd949c23fdcb75fd8136b1fdce54f7f4" Jul 08 22:16:55.199618 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.199436 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 08 22:16:55.199618 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.199515 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 08 22:16:55.199618 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.199677 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f02f34d86a139d7519b22fea82b13f95dd949c23fdcb75fd8136b1fdce54f7f4"} err="failed to get container status \"f02f34d86a139d7519b22fea82b13f95dd949c23fdcb75fd8136b1fdce54f7f4\": rpc error: code = NotFound desc = could not find container \"f02f34d86a139d7519b22fea82b13f95dd949c23fdcb75fd8136b1fdce54f7f4\": container with ID starting with f02f34d86a139d7519b22fea82b13f95dd949c23fdcb75fd8136b1fdce54f7f4 not found: ID does not exist" Jul 08 22:16:55.199618 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.199692 2563 scope.go:117] "RemoveContainer" containerID="2a017ef95710dc8de2a3946e157b9a07080bc59c5ad5f02d1005bb710b45d7b8" Jul 08 22:16:55.200128 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.199713 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 08 22:16:55.200128 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.199722 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 08 22:16:55.200128 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.199787 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 08 22:16:55.200128 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.199865 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-gb5wt\"" Jul 08 22:16:55.200128 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.200087 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a017ef95710dc8de2a3946e157b9a07080bc59c5ad5f02d1005bb710b45d7b8"} err="failed to get container status \"2a017ef95710dc8de2a3946e157b9a07080bc59c5ad5f02d1005bb710b45d7b8\": rpc error: code = NotFound desc = could not find container \"2a017ef95710dc8de2a3946e157b9a07080bc59c5ad5f02d1005bb710b45d7b8\": container with ID starting with 2a017ef95710dc8de2a3946e157b9a07080bc59c5ad5f02d1005bb710b45d7b8 not found: ID does not exist" Jul 08 22:16:55.200128 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.200103 2563 scope.go:117] "RemoveContainer" containerID="ea076cfbd8e0d0031cc9ff72c3fb76fcb4efaa03bfa0ae37ab614269507509a1" Jul 08 22:16:55.200625 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.200353 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea076cfbd8e0d0031cc9ff72c3fb76fcb4efaa03bfa0ae37ab614269507509a1"} err="failed to get container status \"ea076cfbd8e0d0031cc9ff72c3fb76fcb4efaa03bfa0ae37ab614269507509a1\": rpc error: code = NotFound desc = could not find container \"ea076cfbd8e0d0031cc9ff72c3fb76fcb4efaa03bfa0ae37ab614269507509a1\": container with ID starting with ea076cfbd8e0d0031cc9ff72c3fb76fcb4efaa03bfa0ae37ab614269507509a1 not found: ID does not exist" Jul 08 22:16:55.200625 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.200375 2563 scope.go:117] "RemoveContainer" containerID="9e81381c60d659d4968272830a489c51bd947a116e5c7944656bfe9a81c3bfa3" Jul 08 22:16:55.200707 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.200620 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e81381c60d659d4968272830a489c51bd947a116e5c7944656bfe9a81c3bfa3"} err="failed to get container status \"9e81381c60d659d4968272830a489c51bd947a116e5c7944656bfe9a81c3bfa3\": rpc error: code = NotFound desc = could not find container \"9e81381c60d659d4968272830a489c51bd947a116e5c7944656bfe9a81c3bfa3\": container with ID starting with 9e81381c60d659d4968272830a489c51bd947a116e5c7944656bfe9a81c3bfa3 not found: ID does not exist" Jul 08 22:16:55.200707 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.200636 2563 scope.go:117] "RemoveContainer" containerID="31f993706c163fefd47d617879ea96a1d5abe225aea9397c013ffdb0091d0f5d" Jul 08 22:16:55.200947 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.200866 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31f993706c163fefd47d617879ea96a1d5abe225aea9397c013ffdb0091d0f5d"} err="failed to get container status \"31f993706c163fefd47d617879ea96a1d5abe225aea9397c013ffdb0091d0f5d\": rpc error: code = NotFound desc = could not find container \"31f993706c163fefd47d617879ea96a1d5abe225aea9397c013ffdb0091d0f5d\": container with ID starting with 31f993706c163fefd47d617879ea96a1d5abe225aea9397c013ffdb0091d0f5d not found: ID does not exist" Jul 08 22:16:55.200947 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.200902 2563 scope.go:117] "RemoveContainer" containerID="4bfba3ee8d60dc9314129bd39e1b86cbcbc83ebf236b564445829d927908d108" Jul 08 22:16:55.201188 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.201170 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bfba3ee8d60dc9314129bd39e1b86cbcbc83ebf236b564445829d927908d108"} err="failed to get container status \"4bfba3ee8d60dc9314129bd39e1b86cbcbc83ebf236b564445829d927908d108\": rpc error: code = NotFound desc = could not find container \"4bfba3ee8d60dc9314129bd39e1b86cbcbc83ebf236b564445829d927908d108\": container with ID starting with 4bfba3ee8d60dc9314129bd39e1b86cbcbc83ebf236b564445829d927908d108 not found: ID does not exist" Jul 08 22:16:55.201267 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.201188 2563 scope.go:117] "RemoveContainer" containerID="ffe5f3b2d2d8471db7d401688f7a255314d15513f6e2cc336f433bda3baba389" Jul 08 22:16:55.201463 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.201429 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffe5f3b2d2d8471db7d401688f7a255314d15513f6e2cc336f433bda3baba389"} err="failed to get container status \"ffe5f3b2d2d8471db7d401688f7a255314d15513f6e2cc336f433bda3baba389\": rpc error: code = NotFound desc = could not find container \"ffe5f3b2d2d8471db7d401688f7a255314d15513f6e2cc336f433bda3baba389\": container with ID starting with ffe5f3b2d2d8471db7d401688f7a255314d15513f6e2cc336f433bda3baba389 not found: ID does not exist" Jul 08 22:16:55.201463 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.201445 2563 scope.go:117] "RemoveContainer" containerID="f02f34d86a139d7519b22fea82b13f95dd949c23fdcb75fd8136b1fdce54f7f4" Jul 08 22:16:55.201708 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.201689 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f02f34d86a139d7519b22fea82b13f95dd949c23fdcb75fd8136b1fdce54f7f4"} err="failed to get container status \"f02f34d86a139d7519b22fea82b13f95dd949c23fdcb75fd8136b1fdce54f7f4\": rpc error: code = NotFound desc = could not find container \"f02f34d86a139d7519b22fea82b13f95dd949c23fdcb75fd8136b1fdce54f7f4\": container with ID starting with f02f34d86a139d7519b22fea82b13f95dd949c23fdcb75fd8136b1fdce54f7f4 not found: ID does not exist" Jul 08 22:16:55.201708 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.201710 2563 scope.go:117] "RemoveContainer" containerID="2a017ef95710dc8de2a3946e157b9a07080bc59c5ad5f02d1005bb710b45d7b8" Jul 08 22:16:55.201995 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.201954 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a017ef95710dc8de2a3946e157b9a07080bc59c5ad5f02d1005bb710b45d7b8"} err="failed to get container status \"2a017ef95710dc8de2a3946e157b9a07080bc59c5ad5f02d1005bb710b45d7b8\": rpc error: code = NotFound desc = could not find container \"2a017ef95710dc8de2a3946e157b9a07080bc59c5ad5f02d1005bb710b45d7b8\": container with ID starting with 2a017ef95710dc8de2a3946e157b9a07080bc59c5ad5f02d1005bb710b45d7b8 not found: ID does not exist" Jul 08 22:16:55.201995 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.201973 2563 scope.go:117] "RemoveContainer" containerID="ea076cfbd8e0d0031cc9ff72c3fb76fcb4efaa03bfa0ae37ab614269507509a1" Jul 08 22:16:55.202223 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.202193 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea076cfbd8e0d0031cc9ff72c3fb76fcb4efaa03bfa0ae37ab614269507509a1"} err="failed to get container status \"ea076cfbd8e0d0031cc9ff72c3fb76fcb4efaa03bfa0ae37ab614269507509a1\": rpc error: code = NotFound desc = could not find container \"ea076cfbd8e0d0031cc9ff72c3fb76fcb4efaa03bfa0ae37ab614269507509a1\": container with ID starting with ea076cfbd8e0d0031cc9ff72c3fb76fcb4efaa03bfa0ae37ab614269507509a1 not found: ID does not exist" Jul 08 22:16:55.202318 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.202225 2563 scope.go:117] "RemoveContainer" containerID="9e81381c60d659d4968272830a489c51bd947a116e5c7944656bfe9a81c3bfa3" Jul 08 22:16:55.202318 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.202310 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 08 22:16:55.202488 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.202464 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e81381c60d659d4968272830a489c51bd947a116e5c7944656bfe9a81c3bfa3"} err="failed to get container status \"9e81381c60d659d4968272830a489c51bd947a116e5c7944656bfe9a81c3bfa3\": rpc error: code = NotFound desc = could not find container \"9e81381c60d659d4968272830a489c51bd947a116e5c7944656bfe9a81c3bfa3\": container with ID starting with 9e81381c60d659d4968272830a489c51bd947a116e5c7944656bfe9a81c3bfa3 not found: ID does not exist" Jul 08 22:16:55.202542 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.202493 2563 scope.go:117] "RemoveContainer" containerID="31f993706c163fefd47d617879ea96a1d5abe225aea9397c013ffdb0091d0f5d" Jul 08 22:16:55.202868 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.202836 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31f993706c163fefd47d617879ea96a1d5abe225aea9397c013ffdb0091d0f5d"} err="failed to get container status \"31f993706c163fefd47d617879ea96a1d5abe225aea9397c013ffdb0091d0f5d\": rpc error: code = NotFound desc = could not find container \"31f993706c163fefd47d617879ea96a1d5abe225aea9397c013ffdb0091d0f5d\": container with ID starting with 31f993706c163fefd47d617879ea96a1d5abe225aea9397c013ffdb0091d0f5d not found: ID does not exist" Jul 08 22:16:55.202959 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.202870 2563 scope.go:117] "RemoveContainer" containerID="4bfba3ee8d60dc9314129bd39e1b86cbcbc83ebf236b564445829d927908d108" Jul 08 22:16:55.203248 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.203226 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bfba3ee8d60dc9314129bd39e1b86cbcbc83ebf236b564445829d927908d108"} err="failed to get container status \"4bfba3ee8d60dc9314129bd39e1b86cbcbc83ebf236b564445829d927908d108\": rpc error: code = NotFound desc = could not find container \"4bfba3ee8d60dc9314129bd39e1b86cbcbc83ebf236b564445829d927908d108\": container with ID starting with 4bfba3ee8d60dc9314129bd39e1b86cbcbc83ebf236b564445829d927908d108 not found: ID does not exist" Jul 08 22:16:55.203320 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.203250 2563 scope.go:117] "RemoveContainer" containerID="ffe5f3b2d2d8471db7d401688f7a255314d15513f6e2cc336f433bda3baba389" Jul 08 22:16:55.203581 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.203560 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffe5f3b2d2d8471db7d401688f7a255314d15513f6e2cc336f433bda3baba389"} err="failed to get container status \"ffe5f3b2d2d8471db7d401688f7a255314d15513f6e2cc336f433bda3baba389\": rpc error: code = NotFound desc = could not find container \"ffe5f3b2d2d8471db7d401688f7a255314d15513f6e2cc336f433bda3baba389\": container with ID starting with ffe5f3b2d2d8471db7d401688f7a255314d15513f6e2cc336f433bda3baba389 not found: ID does not exist" Jul 08 22:16:55.203581 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.203580 2563 scope.go:117] "RemoveContainer" containerID="f02f34d86a139d7519b22fea82b13f95dd949c23fdcb75fd8136b1fdce54f7f4" Jul 08 22:16:55.203927 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.203867 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f02f34d86a139d7519b22fea82b13f95dd949c23fdcb75fd8136b1fdce54f7f4"} err="failed to get container status \"f02f34d86a139d7519b22fea82b13f95dd949c23fdcb75fd8136b1fdce54f7f4\": rpc error: code = NotFound desc = could not find container \"f02f34d86a139d7519b22fea82b13f95dd949c23fdcb75fd8136b1fdce54f7f4\": container with ID starting with f02f34d86a139d7519b22fea82b13f95dd949c23fdcb75fd8136b1fdce54f7f4 not found: ID does not exist" Jul 08 22:16:55.204021 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.203928 2563 scope.go:117] "RemoveContainer" containerID="2a017ef95710dc8de2a3946e157b9a07080bc59c5ad5f02d1005bb710b45d7b8" Jul 08 22:16:55.204408 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.204385 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a017ef95710dc8de2a3946e157b9a07080bc59c5ad5f02d1005bb710b45d7b8"} err="failed to get container status \"2a017ef95710dc8de2a3946e157b9a07080bc59c5ad5f02d1005bb710b45d7b8\": rpc error: code = NotFound desc = could not find container \"2a017ef95710dc8de2a3946e157b9a07080bc59c5ad5f02d1005bb710b45d7b8\": container with ID starting with 2a017ef95710dc8de2a3946e157b9a07080bc59c5ad5f02d1005bb710b45d7b8 not found: ID does not exist" Jul 08 22:16:55.204497 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.204409 2563 scope.go:117] "RemoveContainer" containerID="ea076cfbd8e0d0031cc9ff72c3fb76fcb4efaa03bfa0ae37ab614269507509a1" Jul 08 22:16:55.204679 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.204652 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea076cfbd8e0d0031cc9ff72c3fb76fcb4efaa03bfa0ae37ab614269507509a1"} err="failed to get container status \"ea076cfbd8e0d0031cc9ff72c3fb76fcb4efaa03bfa0ae37ab614269507509a1\": rpc error: code = NotFound desc = could not find container \"ea076cfbd8e0d0031cc9ff72c3fb76fcb4efaa03bfa0ae37ab614269507509a1\": container with ID starting with ea076cfbd8e0d0031cc9ff72c3fb76fcb4efaa03bfa0ae37ab614269507509a1 not found: ID does not exist" Jul 08 22:16:55.204735 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.204673 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 08 22:16:55.211403 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.211373 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 22:16:55.258783 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.258740 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.258783 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.258783 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.259011 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.258807 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.259011 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.258872 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.259011 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.258938 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-web-config\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.259011 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.258962 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-config-out\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.259011 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.258978 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.259011 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.258999 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.259222 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.259046 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.259222 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.259073 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.259222 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.259103 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.259222 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.259120 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.259222 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.259145 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.259222 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.259170 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.259222 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.259202 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dc5k7\" (UniqueName: \"kubernetes.io/projected/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-kube-api-access-dc5k7\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.259412 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.259226 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-config\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.259412 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.259270 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.259412 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.259292 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.259412 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.259312 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.259412 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.259329 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.317941 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.317839 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35556def-a4de-4ccf-984f-7700074c610c" path="/var/lib/kubelet/pods/35556def-a4de-4ccf-984f-7700074c610c/volumes" Jul 08 22:16:55.360435 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.360400 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.360543 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.360449 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.360543 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.360480 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.360640 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.360609 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.360738 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.360679 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.360738 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.360728 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.360849 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.360796 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dc5k7\" (UniqueName: \"kubernetes.io/projected/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-kube-api-access-dc5k7\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.360849 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.360830 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-config\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.360988 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.360930 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.360988 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.360963 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.361091 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.361020 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.361091 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.361061 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.361187 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.361105 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.361187 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.361133 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.361187 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.361165 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.361338 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.361197 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.361338 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.361245 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-web-config\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.361338 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.361296 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-config-out\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.361338 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.361322 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.361541 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.361351 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.361541 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.361425 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.361636 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.361606 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.362540 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.361907 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.362540 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.362229 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.362540 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.362299 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.363685 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.363657 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-config\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.363859 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.363657 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.364025 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.364003 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.364262 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.364242 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.365100 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.364729 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.365100 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.364993 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.365100 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.365049 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.365298 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.365125 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.365358 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.365343 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.365541 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.365520 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.366209 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.366177 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-config-out\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.366848 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.366825 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-web-config\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.367729 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.367713 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.368107 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.368091 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.370959 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.370936 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dc5k7\" (UniqueName: \"kubernetes.io/projected/cb43ba47-b05e-48e7-aed9-a3849d8ae2ce-kube-api-access-dc5k7\") pod \"prometheus-k8s-0\" (UID: \"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.506968 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.506916 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:16:55.645939 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:55.645908 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 22:16:55.649155 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:16:55.649116 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb43ba47_b05e_48e7_aed9_a3849d8ae2ce.slice/crio-6d83f8a7a6a8f98d5616f1d1b57d84a06a220511915ceea24bc4f8b4654c1d4c WatchSource:0}: Error finding container 6d83f8a7a6a8f98d5616f1d1b57d84a06a220511915ceea24bc4f8b4654c1d4c: Status 404 returned error can't find the container with id 6d83f8a7a6a8f98d5616f1d1b57d84a06a220511915ceea24bc4f8b4654c1d4c Jul 08 22:16:56.119905 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:56.119849 2563 generic.go:356] "Generic (PLEG): container finished" podID="cb43ba47-b05e-48e7-aed9-a3849d8ae2ce" containerID="0e594e9c6a48a03266fc8e6c481aa6ba20cd57543e1480c4afc924eb25851676" exitCode=0 Jul 08 22:16:56.120317 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:56.119951 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce","Type":"ContainerDied","Data":"0e594e9c6a48a03266fc8e6c481aa6ba20cd57543e1480c4afc924eb25851676"} Jul 08 22:16:56.120317 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:56.120020 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce","Type":"ContainerStarted","Data":"6d83f8a7a6a8f98d5616f1d1b57d84a06a220511915ceea24bc4f8b4654c1d4c"} Jul 08 22:16:57.127537 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:57.127501 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce","Type":"ContainerStarted","Data":"49f0532d5faf129bd7060debbf3c11ff230c4fe9e917a5ac3f8f9624d9a4e6c4"} Jul 08 22:16:57.127933 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:57.127603 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce","Type":"ContainerStarted","Data":"ff23bf3d6c1ede76f9b2a576e944099f0f8b58ab480311128eeaf7c17e6106f8"} Jul 08 22:16:57.127933 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:57.127630 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce","Type":"ContainerStarted","Data":"1e00443a5cbf4a8a00d926457b7bb4ce889d91936b609a00f3cd7c444cfb97ce"} Jul 08 22:16:57.127933 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:57.127654 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce","Type":"ContainerStarted","Data":"74813283837e96af6d8f90d03882e8c95dc937e23716676784d596aab271e004"} Jul 08 22:16:57.127933 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:57.127678 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce","Type":"ContainerStarted","Data":"5ba36908c18b540b01e07d228e67395c85e3f78a8d1e8f2c7e7ded2d69b462fd"} Jul 08 22:16:57.127933 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:57.127703 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cb43ba47-b05e-48e7-aed9-a3849d8ae2ce","Type":"ContainerStarted","Data":"e0f4cf000e31e233a866c957ae8717d4302f55369d9a8d43156679ac5e35deb4"} Jul 08 22:16:57.159820 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:16:57.159755 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.140883719 podStartE2EDuration="2.159737468s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.853749ms" isStatefulPod=false podCreationTimestamp="2026-07-08 22:16:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:16:57.156675359 +0000 UTC m=+148.424695388" watchObservedRunningTime="2026-07-08 22:16:57.159737468 +0000 UTC m=+148.427757496" Jul 08 22:17:00.507176 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:00.507145 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:17:06.631099 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:06.631068 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-556d4b65dd-5ljhz"] Jul 08 22:17:06.634167 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:06.634146 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-556d4b65dd-5ljhz" Jul 08 22:17:06.644382 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:06.644360 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-556d4b65dd-5ljhz"] Jul 08 22:17:06.655631 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:06.655600 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/feecbdb8-24ab-4f43-978b-2306c7bb2326-console-config\") pod \"console-556d4b65dd-5ljhz\" (UID: \"feecbdb8-24ab-4f43-978b-2306c7bb2326\") " pod="openshift-console/console-556d4b65dd-5ljhz" Jul 08 22:17:06.655777 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:06.655635 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddbhs\" (UniqueName: \"kubernetes.io/projected/feecbdb8-24ab-4f43-978b-2306c7bb2326-kube-api-access-ddbhs\") pod \"console-556d4b65dd-5ljhz\" (UID: \"feecbdb8-24ab-4f43-978b-2306c7bb2326\") " pod="openshift-console/console-556d4b65dd-5ljhz" Jul 08 22:17:06.655777 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:06.655662 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/feecbdb8-24ab-4f43-978b-2306c7bb2326-service-ca\") pod \"console-556d4b65dd-5ljhz\" (UID: \"feecbdb8-24ab-4f43-978b-2306c7bb2326\") " pod="openshift-console/console-556d4b65dd-5ljhz" Jul 08 22:17:06.655848 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:06.655773 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/feecbdb8-24ab-4f43-978b-2306c7bb2326-oauth-serving-cert\") pod \"console-556d4b65dd-5ljhz\" (UID: \"feecbdb8-24ab-4f43-978b-2306c7bb2326\") " pod="openshift-console/console-556d4b65dd-5ljhz" Jul 08 22:17:06.655848 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:06.655805 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/feecbdb8-24ab-4f43-978b-2306c7bb2326-console-serving-cert\") pod \"console-556d4b65dd-5ljhz\" (UID: \"feecbdb8-24ab-4f43-978b-2306c7bb2326\") " pod="openshift-console/console-556d4b65dd-5ljhz" Jul 08 22:17:06.655848 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:06.655828 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/feecbdb8-24ab-4f43-978b-2306c7bb2326-trusted-ca-bundle\") pod \"console-556d4b65dd-5ljhz\" (UID: \"feecbdb8-24ab-4f43-978b-2306c7bb2326\") " pod="openshift-console/console-556d4b65dd-5ljhz" Jul 08 22:17:06.655993 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:06.655893 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/feecbdb8-24ab-4f43-978b-2306c7bb2326-console-oauth-config\") pod \"console-556d4b65dd-5ljhz\" (UID: \"feecbdb8-24ab-4f43-978b-2306c7bb2326\") " pod="openshift-console/console-556d4b65dd-5ljhz" Jul 08 22:17:06.756871 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:06.756823 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/feecbdb8-24ab-4f43-978b-2306c7bb2326-service-ca\") pod \"console-556d4b65dd-5ljhz\" (UID: \"feecbdb8-24ab-4f43-978b-2306c7bb2326\") " pod="openshift-console/console-556d4b65dd-5ljhz" Jul 08 22:17:06.757098 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:06.756932 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/feecbdb8-24ab-4f43-978b-2306c7bb2326-oauth-serving-cert\") pod \"console-556d4b65dd-5ljhz\" (UID: \"feecbdb8-24ab-4f43-978b-2306c7bb2326\") " pod="openshift-console/console-556d4b65dd-5ljhz" Jul 08 22:17:06.757098 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:06.756965 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/feecbdb8-24ab-4f43-978b-2306c7bb2326-console-serving-cert\") pod \"console-556d4b65dd-5ljhz\" (UID: \"feecbdb8-24ab-4f43-978b-2306c7bb2326\") " pod="openshift-console/console-556d4b65dd-5ljhz" Jul 08 22:17:06.757098 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:06.756981 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/feecbdb8-24ab-4f43-978b-2306c7bb2326-trusted-ca-bundle\") pod \"console-556d4b65dd-5ljhz\" (UID: \"feecbdb8-24ab-4f43-978b-2306c7bb2326\") " pod="openshift-console/console-556d4b65dd-5ljhz" Jul 08 22:17:06.757098 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:06.757034 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/feecbdb8-24ab-4f43-978b-2306c7bb2326-console-oauth-config\") pod \"console-556d4b65dd-5ljhz\" (UID: \"feecbdb8-24ab-4f43-978b-2306c7bb2326\") " pod="openshift-console/console-556d4b65dd-5ljhz" Jul 08 22:17:06.757098 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:06.757071 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/feecbdb8-24ab-4f43-978b-2306c7bb2326-console-config\") pod \"console-556d4b65dd-5ljhz\" (UID: \"feecbdb8-24ab-4f43-978b-2306c7bb2326\") " pod="openshift-console/console-556d4b65dd-5ljhz" Jul 08 22:17:06.757098 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:06.757094 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddbhs\" (UniqueName: \"kubernetes.io/projected/feecbdb8-24ab-4f43-978b-2306c7bb2326-kube-api-access-ddbhs\") pod \"console-556d4b65dd-5ljhz\" (UID: \"feecbdb8-24ab-4f43-978b-2306c7bb2326\") " pod="openshift-console/console-556d4b65dd-5ljhz" Jul 08 22:17:06.757641 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:06.757617 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/feecbdb8-24ab-4f43-978b-2306c7bb2326-oauth-serving-cert\") pod \"console-556d4b65dd-5ljhz\" (UID: \"feecbdb8-24ab-4f43-978b-2306c7bb2326\") " pod="openshift-console/console-556d4b65dd-5ljhz" Jul 08 22:17:06.757742 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:06.757647 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/feecbdb8-24ab-4f43-978b-2306c7bb2326-service-ca\") pod \"console-556d4b65dd-5ljhz\" (UID: \"feecbdb8-24ab-4f43-978b-2306c7bb2326\") " pod="openshift-console/console-556d4b65dd-5ljhz" Jul 08 22:17:06.757861 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:06.757836 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/feecbdb8-24ab-4f43-978b-2306c7bb2326-console-config\") pod \"console-556d4b65dd-5ljhz\" (UID: \"feecbdb8-24ab-4f43-978b-2306c7bb2326\") " pod="openshift-console/console-556d4b65dd-5ljhz" Jul 08 22:17:06.758081 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:06.758060 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/feecbdb8-24ab-4f43-978b-2306c7bb2326-trusted-ca-bundle\") pod \"console-556d4b65dd-5ljhz\" (UID: \"feecbdb8-24ab-4f43-978b-2306c7bb2326\") " pod="openshift-console/console-556d4b65dd-5ljhz" Jul 08 22:17:06.759395 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:06.759367 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/feecbdb8-24ab-4f43-978b-2306c7bb2326-console-oauth-config\") pod \"console-556d4b65dd-5ljhz\" (UID: \"feecbdb8-24ab-4f43-978b-2306c7bb2326\") " pod="openshift-console/console-556d4b65dd-5ljhz" Jul 08 22:17:06.759552 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:06.759535 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/feecbdb8-24ab-4f43-978b-2306c7bb2326-console-serving-cert\") pod \"console-556d4b65dd-5ljhz\" (UID: \"feecbdb8-24ab-4f43-978b-2306c7bb2326\") " pod="openshift-console/console-556d4b65dd-5ljhz" Jul 08 22:17:06.766009 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:06.765989 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddbhs\" (UniqueName: \"kubernetes.io/projected/feecbdb8-24ab-4f43-978b-2306c7bb2326-kube-api-access-ddbhs\") pod \"console-556d4b65dd-5ljhz\" (UID: \"feecbdb8-24ab-4f43-978b-2306c7bb2326\") " pod="openshift-console/console-556d4b65dd-5ljhz" Jul 08 22:17:06.944433 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:06.944327 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-556d4b65dd-5ljhz" Jul 08 22:17:07.079349 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:07.079317 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-556d4b65dd-5ljhz"] Jul 08 22:17:07.082329 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:17:07.082306 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeecbdb8_24ab_4f43_978b_2306c7bb2326.slice/crio-1d5745b68bdf10cff6acde5b868efda4a09c78cae18c6f43cfc3fbea117210cb WatchSource:0}: Error finding container 1d5745b68bdf10cff6acde5b868efda4a09c78cae18c6f43cfc3fbea117210cb: Status 404 returned error can't find the container with id 1d5745b68bdf10cff6acde5b868efda4a09c78cae18c6f43cfc3fbea117210cb Jul 08 22:17:07.168216 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:07.168177 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-556d4b65dd-5ljhz" event={"ID":"feecbdb8-24ab-4f43-978b-2306c7bb2326","Type":"ContainerStarted","Data":"0b392ad79a10fc395afcb9bbb0ef245c934167d28d54c9d801b0623751278ed4"} Jul 08 22:17:07.168413 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:07.168337 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-556d4b65dd-5ljhz" event={"ID":"feecbdb8-24ab-4f43-978b-2306c7bb2326","Type":"ContainerStarted","Data":"1d5745b68bdf10cff6acde5b868efda4a09c78cae18c6f43cfc3fbea117210cb"} Jul 08 22:17:07.186979 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:07.186825 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-556d4b65dd-5ljhz" podStartSLOduration=1.186802144 podStartE2EDuration="1.186802144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 22:17:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:17:07.186185276 +0000 UTC m=+158.454205305" watchObservedRunningTime="2026-07-08 22:17:07.186802144 +0000 UTC m=+158.454822174" Jul 08 22:17:16.944469 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:16.944436 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-556d4b65dd-5ljhz" Jul 08 22:17:16.944969 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:16.944530 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-556d4b65dd-5ljhz" Jul 08 22:17:16.949376 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:16.949349 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-556d4b65dd-5ljhz" Jul 08 22:17:17.209470 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:17.209386 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-556d4b65dd-5ljhz" Jul 08 22:17:17.255114 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:17.255084 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c759449d4-gtwmk"] Jul 08 22:17:42.276129 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:42.276070 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5c759449d4-gtwmk" podUID="aed1b0ef-2682-43b4-af9f-2f3230ece91a" containerName="console" containerID="cri-o://dbfeba2570431e9a11ca197565b50166598c98f7807c8ae14054dc3e7cf3fd32" gracePeriod=15 Jul 08 22:17:42.519764 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:42.519742 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c759449d4-gtwmk" Jul 08 22:17:42.564790 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:42.564707 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aed1b0ef-2682-43b4-af9f-2f3230ece91a-oauth-serving-cert\") pod \"aed1b0ef-2682-43b4-af9f-2f3230ece91a\" (UID: \"aed1b0ef-2682-43b4-af9f-2f3230ece91a\") " Jul 08 22:17:42.564790 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:42.564751 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aed1b0ef-2682-43b4-af9f-2f3230ece91a-console-oauth-config\") pod \"aed1b0ef-2682-43b4-af9f-2f3230ece91a\" (UID: \"aed1b0ef-2682-43b4-af9f-2f3230ece91a\") " Jul 08 22:17:42.564790 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:42.564773 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aed1b0ef-2682-43b4-af9f-2f3230ece91a-trusted-ca-bundle\") pod \"aed1b0ef-2682-43b4-af9f-2f3230ece91a\" (UID: \"aed1b0ef-2682-43b4-af9f-2f3230ece91a\") " Jul 08 22:17:42.565055 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:42.564801 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aed1b0ef-2682-43b4-af9f-2f3230ece91a-console-serving-cert\") pod \"aed1b0ef-2682-43b4-af9f-2f3230ece91a\" (UID: \"aed1b0ef-2682-43b4-af9f-2f3230ece91a\") " Jul 08 22:17:42.565055 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:42.564833 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aed1b0ef-2682-43b4-af9f-2f3230ece91a-service-ca\") pod \"aed1b0ef-2682-43b4-af9f-2f3230ece91a\" (UID: \"aed1b0ef-2682-43b4-af9f-2f3230ece91a\") " Jul 08 22:17:42.565055 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:42.564898 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aed1b0ef-2682-43b4-af9f-2f3230ece91a-console-config\") pod \"aed1b0ef-2682-43b4-af9f-2f3230ece91a\" (UID: \"aed1b0ef-2682-43b4-af9f-2f3230ece91a\") " Jul 08 22:17:42.565210 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:42.565150 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aed1b0ef-2682-43b4-af9f-2f3230ece91a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "aed1b0ef-2682-43b4-af9f-2f3230ece91a" (UID: "aed1b0ef-2682-43b4-af9f-2f3230ece91a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 22:17:42.565283 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:42.565256 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aed1b0ef-2682-43b4-af9f-2f3230ece91a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "aed1b0ef-2682-43b4-af9f-2f3230ece91a" (UID: "aed1b0ef-2682-43b4-af9f-2f3230ece91a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 22:17:42.565378 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:42.565354 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aed1b0ef-2682-43b4-af9f-2f3230ece91a-service-ca" (OuterVolumeSpecName: "service-ca") pod "aed1b0ef-2682-43b4-af9f-2f3230ece91a" (UID: "aed1b0ef-2682-43b4-af9f-2f3230ece91a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 22:17:42.565448 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:42.565370 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aed1b0ef-2682-43b4-af9f-2f3230ece91a-console-config" (OuterVolumeSpecName: "console-config") pod "aed1b0ef-2682-43b4-af9f-2f3230ece91a" (UID: "aed1b0ef-2682-43b4-af9f-2f3230ece91a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 22:17:42.566957 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:42.566934 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aed1b0ef-2682-43b4-af9f-2f3230ece91a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "aed1b0ef-2682-43b4-af9f-2f3230ece91a" (UID: "aed1b0ef-2682-43b4-af9f-2f3230ece91a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 22:17:42.567121 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:42.567100 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aed1b0ef-2682-43b4-af9f-2f3230ece91a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "aed1b0ef-2682-43b4-af9f-2f3230ece91a" (UID: "aed1b0ef-2682-43b4-af9f-2f3230ece91a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 22:17:42.666191 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:42.666157 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2rmld\" (UniqueName: \"kubernetes.io/projected/aed1b0ef-2682-43b4-af9f-2f3230ece91a-kube-api-access-2rmld\") pod \"aed1b0ef-2682-43b4-af9f-2f3230ece91a\" (UID: \"aed1b0ef-2682-43b4-af9f-2f3230ece91a\") " Jul 08 22:17:42.666357 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:42.666345 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aed1b0ef-2682-43b4-af9f-2f3230ece91a-oauth-serving-cert\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:17:42.666398 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:42.666360 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aed1b0ef-2682-43b4-af9f-2f3230ece91a-console-oauth-config\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:17:42.666398 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:42.666374 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aed1b0ef-2682-43b4-af9f-2f3230ece91a-trusted-ca-bundle\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:17:42.666398 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:42.666384 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aed1b0ef-2682-43b4-af9f-2f3230ece91a-console-serving-cert\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:17:42.666398 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:42.666393 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aed1b0ef-2682-43b4-af9f-2f3230ece91a-service-ca\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:17:42.666515 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:42.666404 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aed1b0ef-2682-43b4-af9f-2f3230ece91a-console-config\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:17:42.668209 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:42.668186 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aed1b0ef-2682-43b4-af9f-2f3230ece91a-kube-api-access-2rmld" (OuterVolumeSpecName: "kube-api-access-2rmld") pod "aed1b0ef-2682-43b4-af9f-2f3230ece91a" (UID: "aed1b0ef-2682-43b4-af9f-2f3230ece91a"). InnerVolumeSpecName "kube-api-access-2rmld". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 22:17:42.767177 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:42.767140 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2rmld\" (UniqueName: \"kubernetes.io/projected/aed1b0ef-2682-43b4-af9f-2f3230ece91a-kube-api-access-2rmld\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:17:43.304603 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:43.304569 2563 generic.go:356] "Generic (PLEG): container finished" podID="aed1b0ef-2682-43b4-af9f-2f3230ece91a" containerID="dbfeba2570431e9a11ca197565b50166598c98f7807c8ae14054dc3e7cf3fd32" exitCode=0 Jul 08 22:17:43.305017 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:43.304618 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c759449d4-gtwmk" Jul 08 22:17:43.305017 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:43.304660 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c759449d4-gtwmk" event={"ID":"aed1b0ef-2682-43b4-af9f-2f3230ece91a","Type":"ContainerDied","Data":"dbfeba2570431e9a11ca197565b50166598c98f7807c8ae14054dc3e7cf3fd32"} Jul 08 22:17:43.305017 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:43.304731 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c759449d4-gtwmk" event={"ID":"aed1b0ef-2682-43b4-af9f-2f3230ece91a","Type":"ContainerDied","Data":"017b840a1fab01f332fd05468bbdff84a731a4bf8be2278ffe3e00331b0471a7"} Jul 08 22:17:43.305017 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:43.304752 2563 scope.go:117] "RemoveContainer" containerID="dbfeba2570431e9a11ca197565b50166598c98f7807c8ae14054dc3e7cf3fd32" Jul 08 22:17:43.312749 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:43.312725 2563 scope.go:117] "RemoveContainer" containerID="dbfeba2570431e9a11ca197565b50166598c98f7807c8ae14054dc3e7cf3fd32" Jul 08 22:17:43.313041 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:17:43.313021 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbfeba2570431e9a11ca197565b50166598c98f7807c8ae14054dc3e7cf3fd32\": container with ID starting with dbfeba2570431e9a11ca197565b50166598c98f7807c8ae14054dc3e7cf3fd32 not found: ID does not exist" containerID="dbfeba2570431e9a11ca197565b50166598c98f7807c8ae14054dc3e7cf3fd32" Jul 08 22:17:43.313099 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:43.313050 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbfeba2570431e9a11ca197565b50166598c98f7807c8ae14054dc3e7cf3fd32"} err="failed to get container status \"dbfeba2570431e9a11ca197565b50166598c98f7807c8ae14054dc3e7cf3fd32\": rpc error: code = NotFound desc = could not find container \"dbfeba2570431e9a11ca197565b50166598c98f7807c8ae14054dc3e7cf3fd32\": container with ID starting with dbfeba2570431e9a11ca197565b50166598c98f7807c8ae14054dc3e7cf3fd32 not found: ID does not exist" Jul 08 22:17:43.324699 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:43.324655 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c759449d4-gtwmk"] Jul 08 22:17:43.326519 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:43.326493 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5c759449d4-gtwmk"] Jul 08 22:17:45.316474 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:45.316444 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aed1b0ef-2682-43b4-af9f-2f3230ece91a" path="/var/lib/kubelet/pods/aed1b0ef-2682-43b4-af9f-2f3230ece91a/volumes" Jul 08 22:17:55.507854 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:55.507800 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:17:55.523190 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:55.523164 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:17:56.367554 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:17:56.367526 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 22:18:28.191258 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:18:28.191222 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-56nqt"] Jul 08 22:18:28.191834 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:18:28.191625 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aed1b0ef-2682-43b4-af9f-2f3230ece91a" containerName="console" Jul 08 22:18:28.191834 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:18:28.191642 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="aed1b0ef-2682-43b4-af9f-2f3230ece91a" containerName="console" Jul 08 22:18:28.191834 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:18:28.191710 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="aed1b0ef-2682-43b4-af9f-2f3230ece91a" containerName="console" Jul 08 22:18:28.193894 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:18:28.193868 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-56nqt" Jul 08 22:18:28.196621 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:18:28.196595 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 08 22:18:28.196730 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:18:28.196609 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-5s7qp\"" Jul 08 22:18:28.196730 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:18:28.196612 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 08 22:18:28.204670 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:18:28.204648 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-56nqt"] Jul 08 22:18:28.321756 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:18:28.321726 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxwwh\" (UniqueName: \"kubernetes.io/projected/ec62f837-5ced-4dfe-9a06-c40502593549-kube-api-access-bxwwh\") pod \"cert-manager-55b4d954d5-56nqt\" (UID: \"ec62f837-5ced-4dfe-9a06-c40502593549\") " pod="cert-manager/cert-manager-55b4d954d5-56nqt" Jul 08 22:18:28.321756 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:18:28.321759 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ec62f837-5ced-4dfe-9a06-c40502593549-bound-sa-token\") pod \"cert-manager-55b4d954d5-56nqt\" (UID: \"ec62f837-5ced-4dfe-9a06-c40502593549\") " pod="cert-manager/cert-manager-55b4d954d5-56nqt" Jul 08 22:18:28.422647 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:18:28.422608 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxwwh\" (UniqueName: \"kubernetes.io/projected/ec62f837-5ced-4dfe-9a06-c40502593549-kube-api-access-bxwwh\") pod \"cert-manager-55b4d954d5-56nqt\" (UID: \"ec62f837-5ced-4dfe-9a06-c40502593549\") " pod="cert-manager/cert-manager-55b4d954d5-56nqt" Jul 08 22:18:28.422851 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:18:28.422654 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ec62f837-5ced-4dfe-9a06-c40502593549-bound-sa-token\") pod \"cert-manager-55b4d954d5-56nqt\" (UID: \"ec62f837-5ced-4dfe-9a06-c40502593549\") " pod="cert-manager/cert-manager-55b4d954d5-56nqt" Jul 08 22:18:28.431570 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:18:28.431544 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ec62f837-5ced-4dfe-9a06-c40502593549-bound-sa-token\") pod \"cert-manager-55b4d954d5-56nqt\" (UID: \"ec62f837-5ced-4dfe-9a06-c40502593549\") " pod="cert-manager/cert-manager-55b4d954d5-56nqt" Jul 08 22:18:28.431683 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:18:28.431620 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxwwh\" (UniqueName: \"kubernetes.io/projected/ec62f837-5ced-4dfe-9a06-c40502593549-kube-api-access-bxwwh\") pod \"cert-manager-55b4d954d5-56nqt\" (UID: \"ec62f837-5ced-4dfe-9a06-c40502593549\") " pod="cert-manager/cert-manager-55b4d954d5-56nqt" Jul 08 22:18:28.516012 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:18:28.515982 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-56nqt" Jul 08 22:18:28.643803 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:18:28.643768 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-56nqt"] Jul 08 22:18:28.646721 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:18:28.646694 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podec62f837_5ced_4dfe_9a06_c40502593549.slice/crio-41ec7b8dd03b2ed3b4d35bf710811dd285031013a25755bbe1a50f4a3a90016a WatchSource:0}: Error finding container 41ec7b8dd03b2ed3b4d35bf710811dd285031013a25755bbe1a50f4a3a90016a: Status 404 returned error can't find the container with id 41ec7b8dd03b2ed3b4d35bf710811dd285031013a25755bbe1a50f4a3a90016a Jul 08 22:18:29.469608 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:18:29.469515 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-56nqt" event={"ID":"ec62f837-5ced-4dfe-9a06-c40502593549","Type":"ContainerStarted","Data":"41ec7b8dd03b2ed3b4d35bf710811dd285031013a25755bbe1a50f4a3a90016a"} Jul 08 22:18:32.483300 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:18:32.483265 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-56nqt" event={"ID":"ec62f837-5ced-4dfe-9a06-c40502593549","Type":"ContainerStarted","Data":"0018b4b34c3a89a2770c6f7abda40f3c6d306abc23c1cba1aa34ece47d31ee04"} Jul 08 22:18:32.500940 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:18:32.500842 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-56nqt" podStartSLOduration=1.278069227 podStartE2EDuration="4.50082715s" totalImagesPullingTime="3.222757923s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 22:18:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:18:32.499626071 +0000 UTC m=+243.767646100" watchObservedRunningTime="2026-07-08 22:18:32.50082715 +0000 UTC m=+243.768847179" Jul 08 22:19:15.424954 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:19:15.424850 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kubeflow-trainer-controller-manager-857f77d56c-cwt6j"] Jul 08 22:19:15.430073 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:19:15.430054 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kubeflow-trainer-controller-manager-857f77d56c-cwt6j" Jul 08 22:19:15.433630 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:19:15.433608 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kubeflow-trainer-config\"" Jul 08 22:19:15.433763 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:19:15.433645 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kubeflow-trainer-controller-manager-dockercfg-8dxrd\"" Jul 08 22:19:15.433763 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:19:15.433655 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 08 22:19:15.434742 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:19:15.434725 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kubeflow-trainer-webhook-cert\"" Jul 08 22:19:15.434849 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:19:15.434729 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 08 22:19:15.439405 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:19:15.439384 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kubeflow-trainer-controller-manager-857f77d56c-cwt6j"] Jul 08 22:19:15.469129 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:19:15.469097 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubeflow-trainer-config\" (UniqueName: \"kubernetes.io/configmap/0c930b02-38a6-4b70-9a93-9c6a06b9c7cd-kubeflow-trainer-config\") pod \"kubeflow-trainer-controller-manager-857f77d56c-cwt6j\" (UID: \"0c930b02-38a6-4b70-9a93-9c6a06b9c7cd\") " pod="opendatahub/kubeflow-trainer-controller-manager-857f77d56c-cwt6j" Jul 08 22:19:15.469348 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:19:15.469145 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tq4nq\" (UniqueName: \"kubernetes.io/projected/0c930b02-38a6-4b70-9a93-9c6a06b9c7cd-kube-api-access-tq4nq\") pod \"kubeflow-trainer-controller-manager-857f77d56c-cwt6j\" (UID: \"0c930b02-38a6-4b70-9a93-9c6a06b9c7cd\") " pod="opendatahub/kubeflow-trainer-controller-manager-857f77d56c-cwt6j" Jul 08 22:19:15.469348 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:19:15.469253 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c930b02-38a6-4b70-9a93-9c6a06b9c7cd-cert\") pod \"kubeflow-trainer-controller-manager-857f77d56c-cwt6j\" (UID: \"0c930b02-38a6-4b70-9a93-9c6a06b9c7cd\") " pod="opendatahub/kubeflow-trainer-controller-manager-857f77d56c-cwt6j" Jul 08 22:19:15.570092 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:19:15.570055 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c930b02-38a6-4b70-9a93-9c6a06b9c7cd-cert\") pod \"kubeflow-trainer-controller-manager-857f77d56c-cwt6j\" (UID: \"0c930b02-38a6-4b70-9a93-9c6a06b9c7cd\") " pod="opendatahub/kubeflow-trainer-controller-manager-857f77d56c-cwt6j" Jul 08 22:19:15.570277 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:19:15.570118 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubeflow-trainer-config\" (UniqueName: \"kubernetes.io/configmap/0c930b02-38a6-4b70-9a93-9c6a06b9c7cd-kubeflow-trainer-config\") pod \"kubeflow-trainer-controller-manager-857f77d56c-cwt6j\" (UID: \"0c930b02-38a6-4b70-9a93-9c6a06b9c7cd\") " pod="opendatahub/kubeflow-trainer-controller-manager-857f77d56c-cwt6j" Jul 08 22:19:15.570277 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:19:15.570157 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tq4nq\" (UniqueName: \"kubernetes.io/projected/0c930b02-38a6-4b70-9a93-9c6a06b9c7cd-kube-api-access-tq4nq\") pod \"kubeflow-trainer-controller-manager-857f77d56c-cwt6j\" (UID: \"0c930b02-38a6-4b70-9a93-9c6a06b9c7cd\") " pod="opendatahub/kubeflow-trainer-controller-manager-857f77d56c-cwt6j" Jul 08 22:19:15.570783 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:19:15.570760 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubeflow-trainer-config\" (UniqueName: \"kubernetes.io/configmap/0c930b02-38a6-4b70-9a93-9c6a06b9c7cd-kubeflow-trainer-config\") pod \"kubeflow-trainer-controller-manager-857f77d56c-cwt6j\" (UID: \"0c930b02-38a6-4b70-9a93-9c6a06b9c7cd\") " pod="opendatahub/kubeflow-trainer-controller-manager-857f77d56c-cwt6j" Jul 08 22:19:15.572217 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:19:15.572196 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c930b02-38a6-4b70-9a93-9c6a06b9c7cd-cert\") pod \"kubeflow-trainer-controller-manager-857f77d56c-cwt6j\" (UID: \"0c930b02-38a6-4b70-9a93-9c6a06b9c7cd\") " pod="opendatahub/kubeflow-trainer-controller-manager-857f77d56c-cwt6j" Jul 08 22:19:15.579198 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:19:15.579169 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tq4nq\" (UniqueName: \"kubernetes.io/projected/0c930b02-38a6-4b70-9a93-9c6a06b9c7cd-kube-api-access-tq4nq\") pod \"kubeflow-trainer-controller-manager-857f77d56c-cwt6j\" (UID: \"0c930b02-38a6-4b70-9a93-9c6a06b9c7cd\") " pod="opendatahub/kubeflow-trainer-controller-manager-857f77d56c-cwt6j" Jul 08 22:19:15.739436 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:19:15.739337 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kubeflow-trainer-controller-manager-857f77d56c-cwt6j" Jul 08 22:19:15.868950 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:19:15.868913 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kubeflow-trainer-controller-manager-857f77d56c-cwt6j"] Jul 08 22:19:15.872744 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:19:15.872705 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0c930b02_38a6_4b70_9a93_9c6a06b9c7cd.slice/crio-c5bb49a1144dfaf7f6cee4d7f971c4953bee11f6b2ff6712743dc34699b836a6 WatchSource:0}: Error finding container c5bb49a1144dfaf7f6cee4d7f971c4953bee11f6b2ff6712743dc34699b836a6: Status 404 returned error can't find the container with id c5bb49a1144dfaf7f6cee4d7f971c4953bee11f6b2ff6712743dc34699b836a6 Jul 08 22:19:16.640057 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:19:16.640025 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kubeflow-trainer-controller-manager-857f77d56c-cwt6j" event={"ID":"0c930b02-38a6-4b70-9a93-9c6a06b9c7cd","Type":"ContainerStarted","Data":"c5bb49a1144dfaf7f6cee4d7f971c4953bee11f6b2ff6712743dc34699b836a6"} Jul 08 22:19:18.648500 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:19:18.648452 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kubeflow-trainer-controller-manager-857f77d56c-cwt6j" event={"ID":"0c930b02-38a6-4b70-9a93-9c6a06b9c7cd","Type":"ContainerStarted","Data":"2f5e343c4b241fa3d1dbfa13e5061dcb4c40dcf859f1416633d4733484e8182c"} Jul 08 22:19:18.649007 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:19:18.648585 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kubeflow-trainer-controller-manager-857f77d56c-cwt6j" Jul 08 22:19:18.668236 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:19:18.668174 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kubeflow-trainer-controller-manager-857f77d56c-cwt6j" podStartSLOduration=1.024318808 podStartE2EDuration="3.668123843s" totalImagesPullingTime="2.643805035s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 22:19:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:19:18.66723809 +0000 UTC m=+289.935258119" watchObservedRunningTime="2026-07-08 22:19:18.668123843 +0000 UTC m=+289.936143871" Jul 08 22:19:29.137124 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:19:29.137087 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bltvn_4928f3d7-5f78-4844-9103-cb23d99e9883/ovn-acl-logging/0.log" Jul 08 22:19:29.142722 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:19:29.142697 2563 kubelet.go:1648] "Image garbage collection succeeded" Jul 08 22:19:29.145101 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:19:29.145078 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bltvn_4928f3d7-5f78-4844-9103-cb23d99e9883/ovn-acl-logging/0.log" Jul 08 22:19:34.657790 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:19:34.657759 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kubeflow-trainer-controller-manager-857f77d56c-cwt6j" Jul 08 22:20:44.118189 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:20:44.118153 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-t6rfp/test-trainjob-7svjl-node-0-0-mgsbl"] Jul 08 22:20:44.121488 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:20:44.121469 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-t6rfp/test-trainjob-7svjl-node-0-0-mgsbl" Jul 08 22:20:44.123988 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:20:44.123966 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-t6rfp\"/\"openshift-service-ca.crt\"" Jul 08 22:20:44.124101 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:20:44.123972 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-t6rfp\"/\"default-dockercfg-mlmsp\"" Jul 08 22:20:44.125107 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:20:44.125085 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-t6rfp\"/\"kube-root-ca.crt\"" Jul 08 22:20:44.129424 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:20:44.129400 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-t6rfp/test-trainjob-7svjl-node-0-0-mgsbl"] Jul 08 22:20:44.300426 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:20:44.300395 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dv4qw\" (UniqueName: \"kubernetes.io/projected/2175c1ec-acab-4520-8de6-88cf0fb6a68d-kube-api-access-dv4qw\") pod \"test-trainjob-7svjl-node-0-0-mgsbl\" (UID: \"2175c1ec-acab-4520-8de6-88cf0fb6a68d\") " pod="test-ns-t6rfp/test-trainjob-7svjl-node-0-0-mgsbl" Jul 08 22:20:44.401630 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:20:44.401527 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dv4qw\" (UniqueName: \"kubernetes.io/projected/2175c1ec-acab-4520-8de6-88cf0fb6a68d-kube-api-access-dv4qw\") pod \"test-trainjob-7svjl-node-0-0-mgsbl\" (UID: \"2175c1ec-acab-4520-8de6-88cf0fb6a68d\") " pod="test-ns-t6rfp/test-trainjob-7svjl-node-0-0-mgsbl" Jul 08 22:20:44.411513 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:20:44.411485 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dv4qw\" (UniqueName: \"kubernetes.io/projected/2175c1ec-acab-4520-8de6-88cf0fb6a68d-kube-api-access-dv4qw\") pod \"test-trainjob-7svjl-node-0-0-mgsbl\" (UID: \"2175c1ec-acab-4520-8de6-88cf0fb6a68d\") " pod="test-ns-t6rfp/test-trainjob-7svjl-node-0-0-mgsbl" Jul 08 22:20:44.431327 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:20:44.431301 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-t6rfp/test-trainjob-7svjl-node-0-0-mgsbl" Jul 08 22:20:44.559169 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:20:44.559135 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-t6rfp/test-trainjob-7svjl-node-0-0-mgsbl"] Jul 08 22:20:44.562203 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:20:44.562174 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2175c1ec_acab_4520_8de6_88cf0fb6a68d.slice/crio-55f4a97f98e19bad1ff3c36cea720d91e4821a5fe4b6bc524b2fae85240698c6 WatchSource:0}: Error finding container 55f4a97f98e19bad1ff3c36cea720d91e4821a5fe4b6bc524b2fae85240698c6: Status 404 returned error can't find the container with id 55f4a97f98e19bad1ff3c36cea720d91e4821a5fe4b6bc524b2fae85240698c6 Jul 08 22:20:44.564232 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:20:44.564218 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 22:20:44.975791 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:20:44.975760 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-t6rfp/test-trainjob-7svjl-node-0-0-mgsbl" event={"ID":"2175c1ec-acab-4520-8de6-88cf0fb6a68d","Type":"ContainerStarted","Data":"55f4a97f98e19bad1ff3c36cea720d91e4821a5fe4b6bc524b2fae85240698c6"} Jul 08 22:21:07.361928 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:07.361874 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-66689b576-vpj56"] Jul 08 22:21:07.365269 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:07.365246 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66689b576-vpj56" Jul 08 22:21:07.376382 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:07.376358 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66689b576-vpj56"] Jul 08 22:21:07.392944 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:07.392913 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af8c99fe-183f-4e9f-8ed5-ea4610a36345-console-serving-cert\") pod \"console-66689b576-vpj56\" (UID: \"af8c99fe-183f-4e9f-8ed5-ea4610a36345\") " pod="openshift-console/console-66689b576-vpj56" Jul 08 22:21:07.392944 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:07.392946 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af8c99fe-183f-4e9f-8ed5-ea4610a36345-console-config\") pod \"console-66689b576-vpj56\" (UID: \"af8c99fe-183f-4e9f-8ed5-ea4610a36345\") " pod="openshift-console/console-66689b576-vpj56" Jul 08 22:21:07.393163 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:07.392983 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af8c99fe-183f-4e9f-8ed5-ea4610a36345-trusted-ca-bundle\") pod \"console-66689b576-vpj56\" (UID: \"af8c99fe-183f-4e9f-8ed5-ea4610a36345\") " pod="openshift-console/console-66689b576-vpj56" Jul 08 22:21:07.393163 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:07.393001 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7wr6\" (UniqueName: \"kubernetes.io/projected/af8c99fe-183f-4e9f-8ed5-ea4610a36345-kube-api-access-p7wr6\") pod \"console-66689b576-vpj56\" (UID: \"af8c99fe-183f-4e9f-8ed5-ea4610a36345\") " pod="openshift-console/console-66689b576-vpj56" Jul 08 22:21:07.393163 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:07.393024 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af8c99fe-183f-4e9f-8ed5-ea4610a36345-oauth-serving-cert\") pod \"console-66689b576-vpj56\" (UID: \"af8c99fe-183f-4e9f-8ed5-ea4610a36345\") " pod="openshift-console/console-66689b576-vpj56" Jul 08 22:21:07.393163 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:07.393044 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af8c99fe-183f-4e9f-8ed5-ea4610a36345-console-oauth-config\") pod \"console-66689b576-vpj56\" (UID: \"af8c99fe-183f-4e9f-8ed5-ea4610a36345\") " pod="openshift-console/console-66689b576-vpj56" Jul 08 22:21:07.393163 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:07.393114 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af8c99fe-183f-4e9f-8ed5-ea4610a36345-service-ca\") pod \"console-66689b576-vpj56\" (UID: \"af8c99fe-183f-4e9f-8ed5-ea4610a36345\") " pod="openshift-console/console-66689b576-vpj56" Jul 08 22:21:07.494146 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:07.494105 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af8c99fe-183f-4e9f-8ed5-ea4610a36345-service-ca\") pod \"console-66689b576-vpj56\" (UID: \"af8c99fe-183f-4e9f-8ed5-ea4610a36345\") " pod="openshift-console/console-66689b576-vpj56" Jul 08 22:21:07.494345 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:07.494154 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af8c99fe-183f-4e9f-8ed5-ea4610a36345-console-serving-cert\") pod \"console-66689b576-vpj56\" (UID: \"af8c99fe-183f-4e9f-8ed5-ea4610a36345\") " pod="openshift-console/console-66689b576-vpj56" Jul 08 22:21:07.494345 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:07.494178 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af8c99fe-183f-4e9f-8ed5-ea4610a36345-console-config\") pod \"console-66689b576-vpj56\" (UID: \"af8c99fe-183f-4e9f-8ed5-ea4610a36345\") " pod="openshift-console/console-66689b576-vpj56" Jul 08 22:21:07.494345 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:07.494202 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af8c99fe-183f-4e9f-8ed5-ea4610a36345-trusted-ca-bundle\") pod \"console-66689b576-vpj56\" (UID: \"af8c99fe-183f-4e9f-8ed5-ea4610a36345\") " pod="openshift-console/console-66689b576-vpj56" Jul 08 22:21:07.494345 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:07.494225 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7wr6\" (UniqueName: \"kubernetes.io/projected/af8c99fe-183f-4e9f-8ed5-ea4610a36345-kube-api-access-p7wr6\") pod \"console-66689b576-vpj56\" (UID: \"af8c99fe-183f-4e9f-8ed5-ea4610a36345\") " pod="openshift-console/console-66689b576-vpj56" Jul 08 22:21:07.494345 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:07.494253 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af8c99fe-183f-4e9f-8ed5-ea4610a36345-oauth-serving-cert\") pod \"console-66689b576-vpj56\" (UID: \"af8c99fe-183f-4e9f-8ed5-ea4610a36345\") " pod="openshift-console/console-66689b576-vpj56" Jul 08 22:21:07.494345 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:07.494282 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af8c99fe-183f-4e9f-8ed5-ea4610a36345-console-oauth-config\") pod \"console-66689b576-vpj56\" (UID: \"af8c99fe-183f-4e9f-8ed5-ea4610a36345\") " pod="openshift-console/console-66689b576-vpj56" Jul 08 22:21:07.495021 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:07.494963 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af8c99fe-183f-4e9f-8ed5-ea4610a36345-service-ca\") pod \"console-66689b576-vpj56\" (UID: \"af8c99fe-183f-4e9f-8ed5-ea4610a36345\") " pod="openshift-console/console-66689b576-vpj56" Jul 08 22:21:07.495021 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:07.494999 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af8c99fe-183f-4e9f-8ed5-ea4610a36345-oauth-serving-cert\") pod \"console-66689b576-vpj56\" (UID: \"af8c99fe-183f-4e9f-8ed5-ea4610a36345\") " pod="openshift-console/console-66689b576-vpj56" Jul 08 22:21:07.495021 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:07.495003 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af8c99fe-183f-4e9f-8ed5-ea4610a36345-console-config\") pod \"console-66689b576-vpj56\" (UID: \"af8c99fe-183f-4e9f-8ed5-ea4610a36345\") " pod="openshift-console/console-66689b576-vpj56" Jul 08 22:21:07.495257 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:07.495132 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af8c99fe-183f-4e9f-8ed5-ea4610a36345-trusted-ca-bundle\") pod \"console-66689b576-vpj56\" (UID: \"af8c99fe-183f-4e9f-8ed5-ea4610a36345\") " pod="openshift-console/console-66689b576-vpj56" Jul 08 22:21:07.496687 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:07.496666 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af8c99fe-183f-4e9f-8ed5-ea4610a36345-console-oauth-config\") pod \"console-66689b576-vpj56\" (UID: \"af8c99fe-183f-4e9f-8ed5-ea4610a36345\") " pod="openshift-console/console-66689b576-vpj56" Jul 08 22:21:07.496789 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:07.496759 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af8c99fe-183f-4e9f-8ed5-ea4610a36345-console-serving-cert\") pod \"console-66689b576-vpj56\" (UID: \"af8c99fe-183f-4e9f-8ed5-ea4610a36345\") " pod="openshift-console/console-66689b576-vpj56" Jul 08 22:21:07.504141 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:07.504123 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7wr6\" (UniqueName: \"kubernetes.io/projected/af8c99fe-183f-4e9f-8ed5-ea4610a36345-kube-api-access-p7wr6\") pod \"console-66689b576-vpj56\" (UID: \"af8c99fe-183f-4e9f-8ed5-ea4610a36345\") " pod="openshift-console/console-66689b576-vpj56" Jul 08 22:21:07.675058 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:07.674958 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66689b576-vpj56" Jul 08 22:21:07.803498 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:07.803467 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66689b576-vpj56"] Jul 08 22:21:07.806483 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:21:07.806455 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf8c99fe_183f_4e9f_8ed5_ea4610a36345.slice/crio-b2e0dc2b5fd09fe495265e3b7916424feee628d9a94ce81f95eb36844be9acde WatchSource:0}: Error finding container b2e0dc2b5fd09fe495265e3b7916424feee628d9a94ce81f95eb36844be9acde: Status 404 returned error can't find the container with id b2e0dc2b5fd09fe495265e3b7916424feee628d9a94ce81f95eb36844be9acde Jul 08 22:21:08.077892 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:08.077847 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66689b576-vpj56" event={"ID":"af8c99fe-183f-4e9f-8ed5-ea4610a36345","Type":"ContainerStarted","Data":"0c2fff17d71570da8851653375fd9cd8b044a0396c984e26369ed969b7ebe056"} Jul 08 22:21:08.078085 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:08.077912 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66689b576-vpj56" event={"ID":"af8c99fe-183f-4e9f-8ed5-ea4610a36345","Type":"ContainerStarted","Data":"b2e0dc2b5fd09fe495265e3b7916424feee628d9a94ce81f95eb36844be9acde"} Jul 08 22:21:08.096017 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:08.095965 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-66689b576-vpj56" podStartSLOduration=1.095948345 podStartE2EDuration="1.095948345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 22:21:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:21:08.094668761 +0000 UTC m=+399.362688788" watchObservedRunningTime="2026-07-08 22:21:08.095948345 +0000 UTC m=+399.363968437" Jul 08 22:21:17.675827 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:17.675794 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66689b576-vpj56" Jul 08 22:21:17.676316 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:17.675950 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66689b576-vpj56" Jul 08 22:21:17.681464 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:17.681441 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-66689b576-vpj56" Jul 08 22:21:18.121414 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:18.121386 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-66689b576-vpj56" Jul 08 22:21:18.167353 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:18.167321 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-556d4b65dd-5ljhz"] Jul 08 22:21:43.188562 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:43.188517 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-556d4b65dd-5ljhz" podUID="feecbdb8-24ab-4f43-978b-2306c7bb2326" containerName="console" containerID="cri-o://0b392ad79a10fc395afcb9bbb0ef245c934167d28d54c9d801b0623751278ed4" gracePeriod=15 Jul 08 22:21:43.971671 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:43.971641 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-556d4b65dd-5ljhz" Jul 08 22:21:44.019161 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:44.019128 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ddbhs\" (UniqueName: \"kubernetes.io/projected/feecbdb8-24ab-4f43-978b-2306c7bb2326-kube-api-access-ddbhs\") pod \"feecbdb8-24ab-4f43-978b-2306c7bb2326\" (UID: \"feecbdb8-24ab-4f43-978b-2306c7bb2326\") " Jul 08 22:21:44.019329 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:44.019199 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/feecbdb8-24ab-4f43-978b-2306c7bb2326-console-config\") pod \"feecbdb8-24ab-4f43-978b-2306c7bb2326\" (UID: \"feecbdb8-24ab-4f43-978b-2306c7bb2326\") " Jul 08 22:21:44.019329 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:44.019230 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/feecbdb8-24ab-4f43-978b-2306c7bb2326-service-ca\") pod \"feecbdb8-24ab-4f43-978b-2306c7bb2326\" (UID: \"feecbdb8-24ab-4f43-978b-2306c7bb2326\") " Jul 08 22:21:44.019329 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:44.019258 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/feecbdb8-24ab-4f43-978b-2306c7bb2326-console-serving-cert\") pod \"feecbdb8-24ab-4f43-978b-2306c7bb2326\" (UID: \"feecbdb8-24ab-4f43-978b-2306c7bb2326\") " Jul 08 22:21:44.019329 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:44.019287 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/feecbdb8-24ab-4f43-978b-2306c7bb2326-console-oauth-config\") pod \"feecbdb8-24ab-4f43-978b-2306c7bb2326\" (UID: \"feecbdb8-24ab-4f43-978b-2306c7bb2326\") " Jul 08 22:21:44.019534 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:44.019350 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/feecbdb8-24ab-4f43-978b-2306c7bb2326-trusted-ca-bundle\") pod \"feecbdb8-24ab-4f43-978b-2306c7bb2326\" (UID: \"feecbdb8-24ab-4f43-978b-2306c7bb2326\") " Jul 08 22:21:44.019534 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:44.019378 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/feecbdb8-24ab-4f43-978b-2306c7bb2326-oauth-serving-cert\") pod \"feecbdb8-24ab-4f43-978b-2306c7bb2326\" (UID: \"feecbdb8-24ab-4f43-978b-2306c7bb2326\") " Jul 08 22:21:44.019730 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:44.019695 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/feecbdb8-24ab-4f43-978b-2306c7bb2326-service-ca" (OuterVolumeSpecName: "service-ca") pod "feecbdb8-24ab-4f43-978b-2306c7bb2326" (UID: "feecbdb8-24ab-4f43-978b-2306c7bb2326"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 22:21:44.019863 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:44.019723 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/feecbdb8-24ab-4f43-978b-2306c7bb2326-console-config" (OuterVolumeSpecName: "console-config") pod "feecbdb8-24ab-4f43-978b-2306c7bb2326" (UID: "feecbdb8-24ab-4f43-978b-2306c7bb2326"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 22:21:44.020010 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:44.019879 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/feecbdb8-24ab-4f43-978b-2306c7bb2326-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "feecbdb8-24ab-4f43-978b-2306c7bb2326" (UID: "feecbdb8-24ab-4f43-978b-2306c7bb2326"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 22:21:44.020010 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:44.019959 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/feecbdb8-24ab-4f43-978b-2306c7bb2326-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "feecbdb8-24ab-4f43-978b-2306c7bb2326" (UID: "feecbdb8-24ab-4f43-978b-2306c7bb2326"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 22:21:44.021629 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:44.021599 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/feecbdb8-24ab-4f43-978b-2306c7bb2326-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "feecbdb8-24ab-4f43-978b-2306c7bb2326" (UID: "feecbdb8-24ab-4f43-978b-2306c7bb2326"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 22:21:44.021786 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:44.021749 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/feecbdb8-24ab-4f43-978b-2306c7bb2326-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "feecbdb8-24ab-4f43-978b-2306c7bb2326" (UID: "feecbdb8-24ab-4f43-978b-2306c7bb2326"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 22:21:44.021855 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:44.021798 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/feecbdb8-24ab-4f43-978b-2306c7bb2326-kube-api-access-ddbhs" (OuterVolumeSpecName: "kube-api-access-ddbhs") pod "feecbdb8-24ab-4f43-978b-2306c7bb2326" (UID: "feecbdb8-24ab-4f43-978b-2306c7bb2326"). InnerVolumeSpecName "kube-api-access-ddbhs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 22:21:44.120950 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:44.120855 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/feecbdb8-24ab-4f43-978b-2306c7bb2326-console-config\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:21:44.120950 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:44.120898 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/feecbdb8-24ab-4f43-978b-2306c7bb2326-service-ca\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:21:44.120950 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:44.120907 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/feecbdb8-24ab-4f43-978b-2306c7bb2326-console-serving-cert\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:21:44.120950 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:44.120916 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/feecbdb8-24ab-4f43-978b-2306c7bb2326-console-oauth-config\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:21:44.120950 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:44.120924 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/feecbdb8-24ab-4f43-978b-2306c7bb2326-trusted-ca-bundle\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:21:44.120950 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:44.120932 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/feecbdb8-24ab-4f43-978b-2306c7bb2326-oauth-serving-cert\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:21:44.120950 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:44.120940 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddbhs\" (UniqueName: \"kubernetes.io/projected/feecbdb8-24ab-4f43-978b-2306c7bb2326-kube-api-access-ddbhs\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:21:44.230578 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:44.230541 2563 generic.go:356] "Generic (PLEG): container finished" podID="feecbdb8-24ab-4f43-978b-2306c7bb2326" containerID="0b392ad79a10fc395afcb9bbb0ef245c934167d28d54c9d801b0623751278ed4" exitCode=0 Jul 08 22:21:44.231019 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:44.230593 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-556d4b65dd-5ljhz" Jul 08 22:21:44.231019 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:44.230640 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-556d4b65dd-5ljhz" event={"ID":"feecbdb8-24ab-4f43-978b-2306c7bb2326","Type":"ContainerDied","Data":"0b392ad79a10fc395afcb9bbb0ef245c934167d28d54c9d801b0623751278ed4"} Jul 08 22:21:44.231019 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:44.230706 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-556d4b65dd-5ljhz" event={"ID":"feecbdb8-24ab-4f43-978b-2306c7bb2326","Type":"ContainerDied","Data":"1d5745b68bdf10cff6acde5b868efda4a09c78cae18c6f43cfc3fbea117210cb"} Jul 08 22:21:44.231019 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:44.230739 2563 scope.go:117] "RemoveContainer" containerID="0b392ad79a10fc395afcb9bbb0ef245c934167d28d54c9d801b0623751278ed4" Jul 08 22:21:44.240492 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:44.240469 2563 scope.go:117] "RemoveContainer" containerID="0b392ad79a10fc395afcb9bbb0ef245c934167d28d54c9d801b0623751278ed4" Jul 08 22:21:44.240814 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:21:44.240792 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b392ad79a10fc395afcb9bbb0ef245c934167d28d54c9d801b0623751278ed4\": container with ID starting with 0b392ad79a10fc395afcb9bbb0ef245c934167d28d54c9d801b0623751278ed4 not found: ID does not exist" containerID="0b392ad79a10fc395afcb9bbb0ef245c934167d28d54c9d801b0623751278ed4" Jul 08 22:21:44.240903 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:44.240823 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b392ad79a10fc395afcb9bbb0ef245c934167d28d54c9d801b0623751278ed4"} err="failed to get container status \"0b392ad79a10fc395afcb9bbb0ef245c934167d28d54c9d801b0623751278ed4\": rpc error: code = NotFound desc = could not find container \"0b392ad79a10fc395afcb9bbb0ef245c934167d28d54c9d801b0623751278ed4\": container with ID starting with 0b392ad79a10fc395afcb9bbb0ef245c934167d28d54c9d801b0623751278ed4 not found: ID does not exist" Jul 08 22:21:44.254340 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:44.254308 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-556d4b65dd-5ljhz"] Jul 08 22:21:44.257983 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:44.257952 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-556d4b65dd-5ljhz"] Jul 08 22:21:45.318596 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:21:45.318563 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="feecbdb8-24ab-4f43-978b-2306c7bb2326" path="/var/lib/kubelet/pods/feecbdb8-24ab-4f43-978b-2306c7bb2326/volumes" Jul 08 22:24:29.173030 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:24:29.173001 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bltvn_4928f3d7-5f78-4844-9103-cb23d99e9883/ovn-acl-logging/0.log" Jul 08 22:24:29.176093 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:24:29.176069 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bltvn_4928f3d7-5f78-4844-9103-cb23d99e9883/ovn-acl-logging/0.log" Jul 08 22:28:59.232990 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:28:59.232940 2563 log.go:32] "ListImages with filter from image service failed" err="rpc error: code = DeadlineExceeded desc = context deadline exceeded" filter="" Jul 08 22:28:59.232990 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:28:59.232996 2563 kuberuntime_image.go:108] "Failed to list images" err="rpc error: code = DeadlineExceeded desc = context deadline exceeded" Jul 08 22:28:59.280940 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:28:59.233007 2563 image_gc_manager.go:230] "Failed to update image list" err="rpc error: code = DeadlineExceeded desc = context deadline exceeded" Jul 08 22:31:08.254089 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:08.254003 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bltvn_4928f3d7-5f78-4844-9103-cb23d99e9883/ovn-acl-logging/0.log" Jul 08 22:31:08.254679 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:08.254008 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bltvn_4928f3d7-5f78-4844-9103-cb23d99e9883/ovn-acl-logging/0.log" Jul 08 22:31:09.498381 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:09.498348 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-t6rfp/test-trainjob-7svjl-node-0-0-mgsbl" event={"ID":"2175c1ec-acab-4520-8de6-88cf0fb6a68d","Type":"ContainerStarted","Data":"3c6ebf5cf5b473679bdf24620b78317144f97d497c4b765bbf941b77b5c299f5"} Jul 08 22:31:09.501643 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:09.501624 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-t6rfp\"/\"default-dockercfg-mlmsp\"" Jul 08 22:31:09.523415 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:09.523362 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-t6rfp/test-trainjob-7svjl-node-0-0-mgsbl" podStartSLOduration=0.771920953 podStartE2EDuration="10m25.523347325s" totalImagesPullingTime="10m24.751426372s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 22:20:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:31:09.522292716 +0000 UTC m=+1000.790312744" watchObservedRunningTime="2026-07-08 22:31:09.523347325 +0000 UTC m=+1000.791367353" Jul 08 22:31:09.583873 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:09.583793 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-t6rfp\"/\"kube-root-ca.crt\"" Jul 08 22:31:09.594070 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:09.594040 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-t6rfp\"/\"openshift-service-ca.crt\"" Jul 08 22:31:10.888233 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:10.888208 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-t6rfp_test-trainjob-7svjl-node-0-0-mgsbl_2175c1ec-acab-4520-8de6-88cf0fb6a68d/node/0.log" Jul 08 22:31:13.888015 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:13.887988 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-t6rfp_test-trainjob-7svjl-node-0-0-mgsbl_2175c1ec-acab-4520-8de6-88cf0fb6a68d/node/0.log" Jul 08 22:31:16.887246 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:16.887217 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-t6rfp_test-trainjob-7svjl-node-0-0-mgsbl_2175c1ec-acab-4520-8de6-88cf0fb6a68d/node/0.log" Jul 08 22:31:19.887676 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:19.887645 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-t6rfp_test-trainjob-7svjl-node-0-0-mgsbl_2175c1ec-acab-4520-8de6-88cf0fb6a68d/node/0.log" Jul 08 22:31:22.888744 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:22.888721 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-t6rfp_test-trainjob-7svjl-node-0-0-mgsbl_2175c1ec-acab-4520-8de6-88cf0fb6a68d/node/0.log" Jul 08 22:31:25.887866 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:25.887836 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-t6rfp_test-trainjob-7svjl-node-0-0-mgsbl_2175c1ec-acab-4520-8de6-88cf0fb6a68d/node/0.log" Jul 08 22:31:26.566406 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:26.566371 2563 generic.go:356] "Generic (PLEG): container finished" podID="2175c1ec-acab-4520-8de6-88cf0fb6a68d" containerID="3c6ebf5cf5b473679bdf24620b78317144f97d497c4b765bbf941b77b5c299f5" exitCode=0 Jul 08 22:31:26.566592 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:26.566444 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-t6rfp/test-trainjob-7svjl-node-0-0-mgsbl" event={"ID":"2175c1ec-acab-4520-8de6-88cf0fb6a68d","Type":"ContainerDied","Data":"3c6ebf5cf5b473679bdf24620b78317144f97d497c4b765bbf941b77b5c299f5"} Jul 08 22:31:27.796558 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:27.796525 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-t6rfp/test-trainjob-7svjl-node-0-0-mgsbl" Jul 08 22:31:27.938606 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:27.938523 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dv4qw\" (UniqueName: \"kubernetes.io/projected/2175c1ec-acab-4520-8de6-88cf0fb6a68d-kube-api-access-dv4qw\") pod \"2175c1ec-acab-4520-8de6-88cf0fb6a68d\" (UID: \"2175c1ec-acab-4520-8de6-88cf0fb6a68d\") " Jul 08 22:31:27.940756 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:27.940728 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2175c1ec-acab-4520-8de6-88cf0fb6a68d-kube-api-access-dv4qw" (OuterVolumeSpecName: "kube-api-access-dv4qw") pod "2175c1ec-acab-4520-8de6-88cf0fb6a68d" (UID: "2175c1ec-acab-4520-8de6-88cf0fb6a68d"). InnerVolumeSpecName "kube-api-access-dv4qw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 22:31:28.039491 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:28.039447 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dv4qw\" (UniqueName: \"kubernetes.io/projected/2175c1ec-acab-4520-8de6-88cf0fb6a68d-kube-api-access-dv4qw\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:31:28.574583 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:28.574558 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-t6rfp/test-trainjob-7svjl-node-0-0-mgsbl" Jul 08 22:31:28.574756 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:28.574587 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-t6rfp/test-trainjob-7svjl-node-0-0-mgsbl" event={"ID":"2175c1ec-acab-4520-8de6-88cf0fb6a68d","Type":"ContainerDied","Data":"55f4a97f98e19bad1ff3c36cea720d91e4821a5fe4b6bc524b2fae85240698c6"} Jul 08 22:31:28.574756 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:28.574618 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="55f4a97f98e19bad1ff3c36cea720d91e4821a5fe4b6bc524b2fae85240698c6" Jul 08 22:31:28.775541 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:28.775503 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-t6rfp_test-trainjob-7svjl-node-0-0-mgsbl_2175c1ec-acab-4520-8de6-88cf0fb6a68d/node/0.log" Jul 08 22:31:29.493295 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:29.493257 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-2pn59/must-gather-s6hjb"] Jul 08 22:31:29.493693 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:29.493552 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="feecbdb8-24ab-4f43-978b-2306c7bb2326" containerName="console" Jul 08 22:31:29.493693 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:29.493564 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="feecbdb8-24ab-4f43-978b-2306c7bb2326" containerName="console" Jul 08 22:31:29.493693 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:29.493597 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2175c1ec-acab-4520-8de6-88cf0fb6a68d" containerName="node" Jul 08 22:31:29.493693 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:29.493606 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2175c1ec-acab-4520-8de6-88cf0fb6a68d" containerName="node" Jul 08 22:31:29.493693 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:29.493664 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="feecbdb8-24ab-4f43-978b-2306c7bb2326" containerName="console" Jul 08 22:31:29.493693 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:29.493672 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2175c1ec-acab-4520-8de6-88cf0fb6a68d" containerName="node" Jul 08 22:31:29.556219 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:29.556177 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2pn59/must-gather-s6hjb"] Jul 08 22:31:29.556386 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:29.556282 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2pn59/must-gather-s6hjb" Jul 08 22:31:29.559132 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:29.559107 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2pn59\"/\"openshift-service-ca.crt\"" Jul 08 22:31:29.559277 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:29.559175 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2pn59\"/\"kube-root-ca.crt\"" Jul 08 22:31:29.559277 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:29.559244 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-2pn59\"/\"default-dockercfg-s7q9v\"" Jul 08 22:31:29.650587 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:29.650544 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlxfj\" (UniqueName: \"kubernetes.io/projected/a99f9c49-d0ae-4ced-a6ed-890daed14f08-kube-api-access-qlxfj\") pod \"must-gather-s6hjb\" (UID: \"a99f9c49-d0ae-4ced-a6ed-890daed14f08\") " pod="openshift-must-gather-2pn59/must-gather-s6hjb" Jul 08 22:31:29.650587 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:29.650593 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a99f9c49-d0ae-4ced-a6ed-890daed14f08-must-gather-output\") pod \"must-gather-s6hjb\" (UID: \"a99f9c49-d0ae-4ced-a6ed-890daed14f08\") " pod="openshift-must-gather-2pn59/must-gather-s6hjb" Jul 08 22:31:29.751284 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:29.751196 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlxfj\" (UniqueName: \"kubernetes.io/projected/a99f9c49-d0ae-4ced-a6ed-890daed14f08-kube-api-access-qlxfj\") pod \"must-gather-s6hjb\" (UID: \"a99f9c49-d0ae-4ced-a6ed-890daed14f08\") " pod="openshift-must-gather-2pn59/must-gather-s6hjb" Jul 08 22:31:29.751284 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:29.751242 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a99f9c49-d0ae-4ced-a6ed-890daed14f08-must-gather-output\") pod \"must-gather-s6hjb\" (UID: \"a99f9c49-d0ae-4ced-a6ed-890daed14f08\") " pod="openshift-must-gather-2pn59/must-gather-s6hjb" Jul 08 22:31:29.751548 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:29.751531 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a99f9c49-d0ae-4ced-a6ed-890daed14f08-must-gather-output\") pod \"must-gather-s6hjb\" (UID: \"a99f9c49-d0ae-4ced-a6ed-890daed14f08\") " pod="openshift-must-gather-2pn59/must-gather-s6hjb" Jul 08 22:31:29.760063 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:29.760029 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlxfj\" (UniqueName: \"kubernetes.io/projected/a99f9c49-d0ae-4ced-a6ed-890daed14f08-kube-api-access-qlxfj\") pod \"must-gather-s6hjb\" (UID: \"a99f9c49-d0ae-4ced-a6ed-890daed14f08\") " pod="openshift-must-gather-2pn59/must-gather-s6hjb" Jul 08 22:31:29.865080 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:29.865044 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2pn59/must-gather-s6hjb" Jul 08 22:31:30.124981 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:30.124945 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2pn59/must-gather-s6hjb"] Jul 08 22:31:30.128183 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:31:30.128156 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda99f9c49_d0ae_4ced_a6ed_890daed14f08.slice/crio-0d6606d61d8fa08b25ae22600aa7d63266d43c67b8186fc3feead23db02426e3 WatchSource:0}: Error finding container 0d6606d61d8fa08b25ae22600aa7d63266d43c67b8186fc3feead23db02426e3: Status 404 returned error can't find the container with id 0d6606d61d8fa08b25ae22600aa7d63266d43c67b8186fc3feead23db02426e3 Jul 08 22:31:30.130483 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:30.130467 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 22:31:30.583724 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:30.583688 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2pn59/must-gather-s6hjb" event={"ID":"a99f9c49-d0ae-4ced-a6ed-890daed14f08","Type":"ContainerStarted","Data":"0d6606d61d8fa08b25ae22600aa7d63266d43c67b8186fc3feead23db02426e3"} Jul 08 22:31:33.805973 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:33.805936 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-t6rfp/test-trainjob-7svjl-node-0-0-mgsbl"] Jul 08 22:31:33.808993 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:33.808960 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-t6rfp/test-trainjob-7svjl-node-0-0-mgsbl"] Jul 08 22:31:35.318944 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:35.318908 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2175c1ec-acab-4520-8de6-88cf0fb6a68d" path="/var/lib/kubelet/pods/2175c1ec-acab-4520-8de6-88cf0fb6a68d/volumes" Jul 08 22:31:38.621905 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:38.621694 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2pn59/must-gather-s6hjb" event={"ID":"a99f9c49-d0ae-4ced-a6ed-890daed14f08","Type":"ContainerStarted","Data":"af58589584afdc5e7f8e91ba39896ccb6c0e585c059d66072c0bac76dc95c59d"} Jul 08 22:31:38.621905 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:38.621749 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2pn59/must-gather-s6hjb" event={"ID":"a99f9c49-d0ae-4ced-a6ed-890daed14f08","Type":"ContainerStarted","Data":"ebefc0132f1b1ff3ea1daafd650cddbc30659776cf4cb40dcbe32bed91d9166b"} Jul 08 22:31:38.641331 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:38.641244 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-2pn59/must-gather-s6hjb" podStartSLOduration=2.26677298 podStartE2EDuration="9.641223345s" totalImagesPullingTime="7.374450365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 22:31:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:31:38.640321045 +0000 UTC m=+1029.908341074" watchObservedRunningTime="2026-07-08 22:31:38.641223345 +0000 UTC m=+1029.909243373" Jul 08 22:31:48.798083 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:48.798046 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kubeflow-trainer-controller-manager-857f77d56c-cwt6j_0c930b02-38a6-4b70-9a93-9c6a06b9c7cd/manager/0.log" Jul 08 22:31:49.256342 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:49.256302 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kubeflow-trainer-controller-manager-857f77d56c-cwt6j_0c930b02-38a6-4b70-9a93-9c6a06b9c7cd/manager/0.log" Jul 08 22:31:49.745551 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:31:49.745520 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kubeflow-trainer-controller-manager-857f77d56c-cwt6j_0c930b02-38a6-4b70-9a93-9c6a06b9c7cd/manager/0.log" Jul 08 22:32:27.837375 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:27.837341 2563 generic.go:356] "Generic (PLEG): container finished" podID="a99f9c49-d0ae-4ced-a6ed-890daed14f08" containerID="ebefc0132f1b1ff3ea1daafd650cddbc30659776cf4cb40dcbe32bed91d9166b" exitCode=0 Jul 08 22:32:27.837787 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:27.837410 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2pn59/must-gather-s6hjb" event={"ID":"a99f9c49-d0ae-4ced-a6ed-890daed14f08","Type":"ContainerDied","Data":"ebefc0132f1b1ff3ea1daafd650cddbc30659776cf4cb40dcbe32bed91d9166b"} Jul 08 22:32:27.837787 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:27.837720 2563 scope.go:117] "RemoveContainer" containerID="ebefc0132f1b1ff3ea1daafd650cddbc30659776cf4cb40dcbe32bed91d9166b" Jul 08 22:32:28.410995 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:28.410962 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-2pn59_must-gather-s6hjb_a99f9c49-d0ae-4ced-a6ed-890daed14f08/gather/0.log" Jul 08 22:32:28.914171 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:28.914128 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7xrz2/must-gather-nnvnc"] Jul 08 22:32:28.916552 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:28.916535 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7xrz2/must-gather-nnvnc" Jul 08 22:32:28.919151 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:28.919124 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7xrz2\"/\"openshift-service-ca.crt\"" Jul 08 22:32:28.920269 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:28.920253 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7xrz2\"/\"kube-root-ca.crt\"" Jul 08 22:32:28.920328 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:28.920256 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-7xrz2\"/\"default-dockercfg-fdhgx\"" Jul 08 22:32:28.925017 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:28.924993 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7xrz2/must-gather-nnvnc"] Jul 08 22:32:28.992973 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:28.992943 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/eb99eec6-14d7-42ed-a473-c2e167197d30-must-gather-output\") pod \"must-gather-nnvnc\" (UID: \"eb99eec6-14d7-42ed-a473-c2e167197d30\") " pod="openshift-must-gather-7xrz2/must-gather-nnvnc" Jul 08 22:32:28.992973 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:28.992983 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n48f9\" (UniqueName: \"kubernetes.io/projected/eb99eec6-14d7-42ed-a473-c2e167197d30-kube-api-access-n48f9\") pod \"must-gather-nnvnc\" (UID: \"eb99eec6-14d7-42ed-a473-c2e167197d30\") " pod="openshift-must-gather-7xrz2/must-gather-nnvnc" Jul 08 22:32:29.094254 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:29.094215 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/eb99eec6-14d7-42ed-a473-c2e167197d30-must-gather-output\") pod \"must-gather-nnvnc\" (UID: \"eb99eec6-14d7-42ed-a473-c2e167197d30\") " pod="openshift-must-gather-7xrz2/must-gather-nnvnc" Jul 08 22:32:29.094254 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:29.094262 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n48f9\" (UniqueName: \"kubernetes.io/projected/eb99eec6-14d7-42ed-a473-c2e167197d30-kube-api-access-n48f9\") pod \"must-gather-nnvnc\" (UID: \"eb99eec6-14d7-42ed-a473-c2e167197d30\") " pod="openshift-must-gather-7xrz2/must-gather-nnvnc" Jul 08 22:32:29.094551 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:29.094533 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/eb99eec6-14d7-42ed-a473-c2e167197d30-must-gather-output\") pod \"must-gather-nnvnc\" (UID: \"eb99eec6-14d7-42ed-a473-c2e167197d30\") " pod="openshift-must-gather-7xrz2/must-gather-nnvnc" Jul 08 22:32:29.103352 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:29.103320 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n48f9\" (UniqueName: \"kubernetes.io/projected/eb99eec6-14d7-42ed-a473-c2e167197d30-kube-api-access-n48f9\") pod \"must-gather-nnvnc\" (UID: \"eb99eec6-14d7-42ed-a473-c2e167197d30\") " pod="openshift-must-gather-7xrz2/must-gather-nnvnc" Jul 08 22:32:29.229818 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:29.229740 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-7xrz2\"/\"default-dockercfg-fdhgx\"" Jul 08 22:32:29.237846 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:29.237827 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7xrz2/must-gather-nnvnc" Jul 08 22:32:29.370195 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:29.370171 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7xrz2/must-gather-nnvnc"] Jul 08 22:32:29.372625 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:32:29.372598 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeb99eec6_14d7_42ed_a473_c2e167197d30.slice/crio-ebfdb9bf2452886ba6ec4222753e0d49bbe68c9ce18d900c7d8f7bf9216ae92d WatchSource:0}: Error finding container ebfdb9bf2452886ba6ec4222753e0d49bbe68c9ce18d900c7d8f7bf9216ae92d: Status 404 returned error can't find the container with id ebfdb9bf2452886ba6ec4222753e0d49bbe68c9ce18d900c7d8f7bf9216ae92d Jul 08 22:32:29.847231 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:29.847199 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7xrz2/must-gather-nnvnc" event={"ID":"eb99eec6-14d7-42ed-a473-c2e167197d30","Type":"ContainerStarted","Data":"ebfdb9bf2452886ba6ec4222753e0d49bbe68c9ce18d900c7d8f7bf9216ae92d"} Jul 08 22:32:30.856440 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:30.855935 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7xrz2/must-gather-nnvnc" event={"ID":"eb99eec6-14d7-42ed-a473-c2e167197d30","Type":"ContainerStarted","Data":"c09b446568bd15d7b99e81ffa061b9738c5e75211f4ee0a1079094eedf989fad"} Jul 08 22:32:30.856440 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:30.855992 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7xrz2/must-gather-nnvnc" event={"ID":"eb99eec6-14d7-42ed-a473-c2e167197d30","Type":"ContainerStarted","Data":"4890a26a960f15eda2d52fd2ed4da80ee47b2987ad51ac5ecedd6385c8828f61"} Jul 08 22:32:30.873362 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:30.873296 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-7xrz2/must-gather-nnvnc" podStartSLOduration=2.016789971 podStartE2EDuration="2.873274688s" totalImagesPullingTime="856.484717ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 22:32:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:32:30.87046469 +0000 UTC m=+1082.138484724" watchObservedRunningTime="2026-07-08 22:32:30.873274688 +0000 UTC m=+1082.141294719" Jul 08 22:32:31.818993 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:31.818949 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-fffds_4a354dc2-56c0-4280-96d1-b560d70be2b2/global-pull-secret-syncer/0.log" Jul 08 22:32:31.884322 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:31.884291 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-p4pz6_e1de91b1-aa3f-4288-9b87-718980dcea7e/konnectivity-agent/0.log" Jul 08 22:32:31.983805 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:31.983764 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-162.ec2.internal_55e574bf7ca2ba660429fe3c53ea934c/haproxy/0.log" Jul 08 22:32:33.753695 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:33.753584 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-2pn59/must-gather-s6hjb"] Jul 08 22:32:33.754190 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:33.753860 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-must-gather-2pn59/must-gather-s6hjb" podUID="a99f9c49-d0ae-4ced-a6ed-890daed14f08" containerName="copy" containerID="cri-o://af58589584afdc5e7f8e91ba39896ccb6c0e585c059d66072c0bac76dc95c59d" gracePeriod=2 Jul 08 22:32:33.756301 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:32:33.756263 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-s6hjb\" is forbidden: User \"system:node:ip-10-0-133-162.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-2pn59\": no relationship found between node 'ip-10-0-133-162.ec2.internal' and this object" podUID="a99f9c49-d0ae-4ced-a6ed-890daed14f08" pod="openshift-must-gather-2pn59/must-gather-s6hjb" Jul 08 22:32:33.757188 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:33.757153 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-2pn59/must-gather-s6hjb"] Jul 08 22:32:33.875543 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:33.875443 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-2pn59_must-gather-s6hjb_a99f9c49-d0ae-4ced-a6ed-890daed14f08/copy/0.log" Jul 08 22:32:33.876498 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:33.876379 2563 generic.go:356] "Generic (PLEG): container finished" podID="a99f9c49-d0ae-4ced-a6ed-890daed14f08" containerID="af58589584afdc5e7f8e91ba39896ccb6c0e585c059d66072c0bac76dc95c59d" exitCode=143 Jul 08 22:32:34.130281 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:34.130011 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-2pn59_must-gather-s6hjb_a99f9c49-d0ae-4ced-a6ed-890daed14f08/copy/0.log" Jul 08 22:32:34.135232 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:34.134989 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2pn59/must-gather-s6hjb" Jul 08 22:32:34.137969 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:32:34.137435 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-s6hjb\" is forbidden: User \"system:node:ip-10-0-133-162.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-2pn59\": no relationship found between node 'ip-10-0-133-162.ec2.internal' and this object" podUID="a99f9c49-d0ae-4ced-a6ed-890daed14f08" pod="openshift-must-gather-2pn59/must-gather-s6hjb" Jul 08 22:32:34.151905 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:34.151157 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qlxfj\" (UniqueName: \"kubernetes.io/projected/a99f9c49-d0ae-4ced-a6ed-890daed14f08-kube-api-access-qlxfj\") pod \"a99f9c49-d0ae-4ced-a6ed-890daed14f08\" (UID: \"a99f9c49-d0ae-4ced-a6ed-890daed14f08\") " Jul 08 22:32:34.151905 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:34.151220 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a99f9c49-d0ae-4ced-a6ed-890daed14f08-must-gather-output\") pod \"a99f9c49-d0ae-4ced-a6ed-890daed14f08\" (UID: \"a99f9c49-d0ae-4ced-a6ed-890daed14f08\") " Jul 08 22:32:34.155796 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:34.153382 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a99f9c49-d0ae-4ced-a6ed-890daed14f08-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "a99f9c49-d0ae-4ced-a6ed-890daed14f08" (UID: "a99f9c49-d0ae-4ced-a6ed-890daed14f08"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:32:34.159901 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:34.159043 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a99f9c49-d0ae-4ced-a6ed-890daed14f08-kube-api-access-qlxfj" (OuterVolumeSpecName: "kube-api-access-qlxfj") pod "a99f9c49-d0ae-4ced-a6ed-890daed14f08" (UID: "a99f9c49-d0ae-4ced-a6ed-890daed14f08"). InnerVolumeSpecName "kube-api-access-qlxfj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 22:32:34.252980 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:34.252940 2563 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a99f9c49-d0ae-4ced-a6ed-890daed14f08-must-gather-output\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:32:34.252980 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:34.252981 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qlxfj\" (UniqueName: \"kubernetes.io/projected/a99f9c49-d0ae-4ced-a6ed-890daed14f08-kube-api-access-qlxfj\") on node \"ip-10-0-133-162.ec2.internal\" DevicePath \"\"" Jul 08 22:32:34.885235 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:34.885144 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-2pn59_must-gather-s6hjb_a99f9c49-d0ae-4ced-a6ed-890daed14f08/copy/0.log" Jul 08 22:32:34.889922 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:34.886242 2563 scope.go:117] "RemoveContainer" containerID="af58589584afdc5e7f8e91ba39896ccb6c0e585c059d66072c0bac76dc95c59d" Jul 08 22:32:34.889922 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:34.886411 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2pn59/must-gather-s6hjb" Jul 08 22:32:34.893255 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:32:34.893219 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-s6hjb\" is forbidden: User \"system:node:ip-10-0-133-162.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-2pn59\": no relationship found between node 'ip-10-0-133-162.ec2.internal' and this object" podUID="a99f9c49-d0ae-4ced-a6ed-890daed14f08" pod="openshift-must-gather-2pn59/must-gather-s6hjb" Jul 08 22:32:34.909531 ip-10-0-133-162 kubenswrapper[2563]: E0708 22:32:34.909489 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-s6hjb\" is forbidden: User \"system:node:ip-10-0-133-162.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-2pn59\": no relationship found between node 'ip-10-0-133-162.ec2.internal' and this object" podUID="a99f9c49-d0ae-4ced-a6ed-890daed14f08" pod="openshift-must-gather-2pn59/must-gather-s6hjb" Jul 08 22:32:34.912601 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:34.912500 2563 scope.go:117] "RemoveContainer" containerID="ebefc0132f1b1ff3ea1daafd650cddbc30659776cf4cb40dcbe32bed91d9166b" Jul 08 22:32:35.320869 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:35.320831 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a99f9c49-d0ae-4ced-a6ed-890daed14f08" path="/var/lib/kubelet/pods/a99f9c49-d0ae-4ced-a6ed-890daed14f08/volumes" Jul 08 22:32:36.103951 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:36.103861 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-kqq85_9ba2c75b-169e-4f62-85a4-34171290a5bf/kube-state-metrics/0.log" Jul 08 22:32:36.130576 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:36.130545 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-kqq85_9ba2c75b-169e-4f62-85a4-34171290a5bf/kube-rbac-proxy-main/0.log" Jul 08 22:32:36.161520 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:36.161492 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-kqq85_9ba2c75b-169e-4f62-85a4-34171290a5bf/kube-rbac-proxy-self/0.log" Jul 08 22:32:36.229774 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:36.229742 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-thl2v_f38983c4-b714-4da9-9336-28b1da2c78b5/monitoring-plugin/0.log" Jul 08 22:32:36.269053 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:36.269028 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-46rlm_eb2520ae-ada6-4124-ae5c-8a7bd461d500/node-exporter/0.log" Jul 08 22:32:36.300568 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:36.300516 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-46rlm_eb2520ae-ada6-4124-ae5c-8a7bd461d500/kube-rbac-proxy/0.log" Jul 08 22:32:36.324736 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:36.324706 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-46rlm_eb2520ae-ada6-4124-ae5c-8a7bd461d500/init-textfile/0.log" Jul 08 22:32:36.514061 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:36.514003 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-jcw5n_5085739c-ac1f-4818-9a2a-065a2bc6cab1/kube-rbac-proxy-main/0.log" Jul 08 22:32:36.544442 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:36.544412 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-jcw5n_5085739c-ac1f-4818-9a2a-065a2bc6cab1/kube-rbac-proxy-self/0.log" Jul 08 22:32:36.567328 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:36.567276 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-jcw5n_5085739c-ac1f-4818-9a2a-065a2bc6cab1/openshift-state-metrics/0.log" Jul 08 22:32:36.607672 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:36.607638 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cb43ba47-b05e-48e7-aed9-a3849d8ae2ce/prometheus/0.log" Jul 08 22:32:36.633369 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:36.633340 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cb43ba47-b05e-48e7-aed9-a3849d8ae2ce/config-reloader/0.log" Jul 08 22:32:36.658267 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:36.658239 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cb43ba47-b05e-48e7-aed9-a3849d8ae2ce/thanos-sidecar/0.log" Jul 08 22:32:36.680871 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:36.680841 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cb43ba47-b05e-48e7-aed9-a3849d8ae2ce/kube-rbac-proxy-web/0.log" Jul 08 22:32:36.707758 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:36.707728 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cb43ba47-b05e-48e7-aed9-a3849d8ae2ce/kube-rbac-proxy/0.log" Jul 08 22:32:36.732428 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:36.732398 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cb43ba47-b05e-48e7-aed9-a3849d8ae2ce/kube-rbac-proxy-thanos/0.log" Jul 08 22:32:36.756516 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:36.756491 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cb43ba47-b05e-48e7-aed9-a3849d8ae2ce/init-config-reloader/0.log" Jul 08 22:32:36.788470 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:36.788379 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-w2c67_67767774-0fce-4cea-b8ba-3906c5af3293/prometheus-operator/0.log" Jul 08 22:32:36.811235 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:36.811208 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-w2c67_67767774-0fce-4cea-b8ba-3906c5af3293/kube-rbac-proxy/0.log" Jul 08 22:32:39.230514 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:39.230440 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-66689b576-vpj56_af8c99fe-183f-4e9f-8ed5-ea4610a36345/console/0.log" Jul 08 22:32:39.902452 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:39.902420 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7xrz2/perf-node-gather-daemonset-p5qf7"] Jul 08 22:32:39.902746 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:39.902733 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a99f9c49-d0ae-4ced-a6ed-890daed14f08" containerName="copy" Jul 08 22:32:39.902746 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:39.902746 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a99f9c49-d0ae-4ced-a6ed-890daed14f08" containerName="copy" Jul 08 22:32:39.902860 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:39.902759 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a99f9c49-d0ae-4ced-a6ed-890daed14f08" containerName="gather" Jul 08 22:32:39.902860 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:39.902764 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a99f9c49-d0ae-4ced-a6ed-890daed14f08" containerName="gather" Jul 08 22:32:39.902860 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:39.902813 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a99f9c49-d0ae-4ced-a6ed-890daed14f08" containerName="gather" Jul 08 22:32:39.902860 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:39.902820 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a99f9c49-d0ae-4ced-a6ed-890daed14f08" containerName="copy" Jul 08 22:32:39.906478 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:39.906455 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7xrz2/perf-node-gather-daemonset-p5qf7" Jul 08 22:32:39.917378 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:39.917351 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7xrz2/perf-node-gather-daemonset-p5qf7"] Jul 08 22:32:40.005127 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:40.005095 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/da61584d-6b1a-417b-95e8-4060bbbac047-proc\") pod \"perf-node-gather-daemonset-p5qf7\" (UID: \"da61584d-6b1a-417b-95e8-4060bbbac047\") " pod="openshift-must-gather-7xrz2/perf-node-gather-daemonset-p5qf7" Jul 08 22:32:40.005312 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:40.005133 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/da61584d-6b1a-417b-95e8-4060bbbac047-lib-modules\") pod \"perf-node-gather-daemonset-p5qf7\" (UID: \"da61584d-6b1a-417b-95e8-4060bbbac047\") " pod="openshift-must-gather-7xrz2/perf-node-gather-daemonset-p5qf7" Jul 08 22:32:40.005312 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:40.005158 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/da61584d-6b1a-417b-95e8-4060bbbac047-podres\") pod \"perf-node-gather-daemonset-p5qf7\" (UID: \"da61584d-6b1a-417b-95e8-4060bbbac047\") " pod="openshift-must-gather-7xrz2/perf-node-gather-daemonset-p5qf7" Jul 08 22:32:40.005312 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:40.005236 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da61584d-6b1a-417b-95e8-4060bbbac047-sys\") pod \"perf-node-gather-daemonset-p5qf7\" (UID: \"da61584d-6b1a-417b-95e8-4060bbbac047\") " pod="openshift-must-gather-7xrz2/perf-node-gather-daemonset-p5qf7" Jul 08 22:32:40.005312 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:40.005281 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpxm7\" (UniqueName: \"kubernetes.io/projected/da61584d-6b1a-417b-95e8-4060bbbac047-kube-api-access-jpxm7\") pod \"perf-node-gather-daemonset-p5qf7\" (UID: \"da61584d-6b1a-417b-95e8-4060bbbac047\") " pod="openshift-must-gather-7xrz2/perf-node-gather-daemonset-p5qf7" Jul 08 22:32:40.106371 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:40.106332 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/da61584d-6b1a-417b-95e8-4060bbbac047-proc\") pod \"perf-node-gather-daemonset-p5qf7\" (UID: \"da61584d-6b1a-417b-95e8-4060bbbac047\") " pod="openshift-must-gather-7xrz2/perf-node-gather-daemonset-p5qf7" Jul 08 22:32:40.106553 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:40.106385 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/da61584d-6b1a-417b-95e8-4060bbbac047-lib-modules\") pod \"perf-node-gather-daemonset-p5qf7\" (UID: \"da61584d-6b1a-417b-95e8-4060bbbac047\") " pod="openshift-must-gather-7xrz2/perf-node-gather-daemonset-p5qf7" Jul 08 22:32:40.106553 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:40.106421 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/da61584d-6b1a-417b-95e8-4060bbbac047-podres\") pod \"perf-node-gather-daemonset-p5qf7\" (UID: \"da61584d-6b1a-417b-95e8-4060bbbac047\") " pod="openshift-must-gather-7xrz2/perf-node-gather-daemonset-p5qf7" Jul 08 22:32:40.106553 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:40.106451 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da61584d-6b1a-417b-95e8-4060bbbac047-sys\") pod \"perf-node-gather-daemonset-p5qf7\" (UID: \"da61584d-6b1a-417b-95e8-4060bbbac047\") " pod="openshift-must-gather-7xrz2/perf-node-gather-daemonset-p5qf7" Jul 08 22:32:40.106553 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:40.106461 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/da61584d-6b1a-417b-95e8-4060bbbac047-proc\") pod \"perf-node-gather-daemonset-p5qf7\" (UID: \"da61584d-6b1a-417b-95e8-4060bbbac047\") " pod="openshift-must-gather-7xrz2/perf-node-gather-daemonset-p5qf7" Jul 08 22:32:40.106553 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:40.106522 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpxm7\" (UniqueName: \"kubernetes.io/projected/da61584d-6b1a-417b-95e8-4060bbbac047-kube-api-access-jpxm7\") pod \"perf-node-gather-daemonset-p5qf7\" (UID: \"da61584d-6b1a-417b-95e8-4060bbbac047\") " pod="openshift-must-gather-7xrz2/perf-node-gather-daemonset-p5qf7" Jul 08 22:32:40.106553 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:40.106532 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da61584d-6b1a-417b-95e8-4060bbbac047-sys\") pod \"perf-node-gather-daemonset-p5qf7\" (UID: \"da61584d-6b1a-417b-95e8-4060bbbac047\") " pod="openshift-must-gather-7xrz2/perf-node-gather-daemonset-p5qf7" Jul 08 22:32:40.106786 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:40.106580 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/da61584d-6b1a-417b-95e8-4060bbbac047-lib-modules\") pod \"perf-node-gather-daemonset-p5qf7\" (UID: \"da61584d-6b1a-417b-95e8-4060bbbac047\") " pod="openshift-must-gather-7xrz2/perf-node-gather-daemonset-p5qf7" Jul 08 22:32:40.106786 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:40.106585 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/da61584d-6b1a-417b-95e8-4060bbbac047-podres\") pod \"perf-node-gather-daemonset-p5qf7\" (UID: \"da61584d-6b1a-417b-95e8-4060bbbac047\") " pod="openshift-must-gather-7xrz2/perf-node-gather-daemonset-p5qf7" Jul 08 22:32:40.115387 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:40.115359 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpxm7\" (UniqueName: \"kubernetes.io/projected/da61584d-6b1a-417b-95e8-4060bbbac047-kube-api-access-jpxm7\") pod \"perf-node-gather-daemonset-p5qf7\" (UID: \"da61584d-6b1a-417b-95e8-4060bbbac047\") " pod="openshift-must-gather-7xrz2/perf-node-gather-daemonset-p5qf7" Jul 08 22:32:40.218551 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:40.218464 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7xrz2/perf-node-gather-daemonset-p5qf7" Jul 08 22:32:40.369661 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:40.369623 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7xrz2/perf-node-gather-daemonset-p5qf7"] Jul 08 22:32:40.372829 ip-10-0-133-162 kubenswrapper[2563]: W0708 22:32:40.372807 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podda61584d_6b1a_417b_95e8_4060bbbac047.slice/crio-1ce6adc31f48223ba349f12aa9786773318bfa6229461a2f50234be1286ce693 WatchSource:0}: Error finding container 1ce6adc31f48223ba349f12aa9786773318bfa6229461a2f50234be1286ce693: Status 404 returned error can't find the container with id 1ce6adc31f48223ba349f12aa9786773318bfa6229461a2f50234be1286ce693 Jul 08 22:32:40.634941 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:40.634912 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vrwxr_90dd30c2-b9af-4b4e-8f07-093761fdfeaf/dns/0.log" Jul 08 22:32:40.653913 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:40.653871 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vrwxr_90dd30c2-b9af-4b4e-8f07-093761fdfeaf/kube-rbac-proxy/0.log" Jul 08 22:32:40.675107 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:40.675075 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mzzbg_b5a657a4-5088-4c35-96dc-78d8b4a9289f/dns-node-resolver/0.log" Jul 08 22:32:40.921208 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:40.921173 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7xrz2/perf-node-gather-daemonset-p5qf7" event={"ID":"da61584d-6b1a-417b-95e8-4060bbbac047","Type":"ContainerStarted","Data":"65069a00780e555c3d5d582fdfdee8bd8f5623f1cbd428ccf01f50367119aea3"} Jul 08 22:32:40.921482 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:40.921460 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7xrz2/perf-node-gather-daemonset-p5qf7" event={"ID":"da61584d-6b1a-417b-95e8-4060bbbac047","Type":"ContainerStarted","Data":"1ce6adc31f48223ba349f12aa9786773318bfa6229461a2f50234be1286ce693"} Jul 08 22:32:40.921628 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:40.921614 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-7xrz2/perf-node-gather-daemonset-p5qf7" Jul 08 22:32:40.938747 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:40.938698 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-7xrz2/perf-node-gather-daemonset-p5qf7" podStartSLOduration=1.9386804469999999 podStartE2EDuration="1.938680447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 22:32:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:32:40.93818795 +0000 UTC m=+1092.206207990" watchObservedRunningTime="2026-07-08 22:32:40.938680447 +0000 UTC m=+1092.206700474" Jul 08 22:32:41.197165 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:41.197088 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4rvb8_b0e80baf-e49e-4b91-ab27-dbc8dc3c2dc4/node-ca/0.log" Jul 08 22:32:42.245187 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:42.245162 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jwrdk_7b839249-f20b-4b16-a761-744920da6bd0/serve-healthcheck-canary/0.log" Jul 08 22:32:42.592107 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:42.592029 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bsczr_a9d99b14-d934-42dc-ba6a-a16effd31a85/kube-rbac-proxy/0.log" Jul 08 22:32:42.610594 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:42.610566 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bsczr_a9d99b14-d934-42dc-ba6a-a16effd31a85/exporter/0.log" Jul 08 22:32:42.631916 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:42.631849 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bsczr_a9d99b14-d934-42dc-ba6a-a16effd31a85/extractor/0.log" Jul 08 22:32:47.943198 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:47.943161 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-7xrz2/perf-node-gather-daemonset-p5qf7" Jul 08 22:32:48.453440 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:48.453409 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-8fgq2_3dbef84e-5232-42ba-9727-c632df781129/kube-multus/0.log" Jul 08 22:32:48.780257 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:48.780232 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fbm5c_e722d997-2e47-40ac-bb5c-48e25dde7cfd/kube-multus-additional-cni-plugins/0.log" Jul 08 22:32:48.800060 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:48.800027 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fbm5c_e722d997-2e47-40ac-bb5c-48e25dde7cfd/egress-router-binary-copy/0.log" Jul 08 22:32:48.819086 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:48.819059 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fbm5c_e722d997-2e47-40ac-bb5c-48e25dde7cfd/cni-plugins/0.log" Jul 08 22:32:48.838921 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:48.838870 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fbm5c_e722d997-2e47-40ac-bb5c-48e25dde7cfd/bond-cni-plugin/0.log" Jul 08 22:32:48.858520 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:48.858491 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fbm5c_e722d997-2e47-40ac-bb5c-48e25dde7cfd/routeoverride-cni/0.log" Jul 08 22:32:48.877436 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:48.877411 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fbm5c_e722d997-2e47-40ac-bb5c-48e25dde7cfd/whereabouts-cni-bincopy/0.log" Jul 08 22:32:48.896255 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:48.896227 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fbm5c_e722d997-2e47-40ac-bb5c-48e25dde7cfd/whereabouts-cni/0.log" Jul 08 22:32:48.974895 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:48.974847 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-jhpj6_7c5c6b58-e07d-4d72-af33-ebe169402d4e/network-metrics-daemon/0.log" Jul 08 22:32:48.993417 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:48.993392 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-jhpj6_7c5c6b58-e07d-4d72-af33-ebe169402d4e/kube-rbac-proxy/0.log" Jul 08 22:32:50.090676 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:50.090645 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bltvn_4928f3d7-5f78-4844-9103-cb23d99e9883/ovn-controller/0.log" Jul 08 22:32:50.107329 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:50.107297 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bltvn_4928f3d7-5f78-4844-9103-cb23d99e9883/ovn-acl-logging/1.log" Jul 08 22:32:50.107516 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:50.107453 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bltvn_4928f3d7-5f78-4844-9103-cb23d99e9883/ovn-acl-logging/0.log" Jul 08 22:32:50.128563 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:50.128534 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bltvn_4928f3d7-5f78-4844-9103-cb23d99e9883/kube-rbac-proxy-node/0.log" Jul 08 22:32:50.154379 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:50.154334 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bltvn_4928f3d7-5f78-4844-9103-cb23d99e9883/kube-rbac-proxy-ovn-metrics/0.log" Jul 08 22:32:50.175704 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:50.175676 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bltvn_4928f3d7-5f78-4844-9103-cb23d99e9883/northd/0.log" Jul 08 22:32:50.202634 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:50.202607 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bltvn_4928f3d7-5f78-4844-9103-cb23d99e9883/nbdb/0.log" Jul 08 22:32:50.221904 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:50.221862 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bltvn_4928f3d7-5f78-4844-9103-cb23d99e9883/sbdb/0.log" Jul 08 22:32:50.281867 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:50.281837 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bltvn_4928f3d7-5f78-4844-9103-cb23d99e9883/ovnkube-controller/0.log" Jul 08 22:32:51.505963 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:51.505933 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-98x2n_6a151365-c622-4361-862f-ed202f3045f7/network-check-target-container/0.log" Jul 08 22:32:52.414788 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:52.414756 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-45nvq_c8c71659-1efa-481c-9524-2ebf0b684b97/iptables-alerter/0.log" Jul 08 22:32:53.022072 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:53.022045 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-bj85s_15388137-d252-4a28-996c-3f0ca3236e35/tuned/0.log" Jul 08 22:32:56.554561 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:56.554514 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-vfvvt_ec6d07d3-a0df-413e-abcb-f87ffc795c7d/csi-driver/0.log" Jul 08 22:32:56.576561 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:56.576535 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-vfvvt_ec6d07d3-a0df-413e-abcb-f87ffc795c7d/csi-node-driver-registrar/0.log" Jul 08 22:32:56.595083 ip-10-0-133-162 kubenswrapper[2563]: I0708 22:32:56.595058 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-vfvvt_ec6d07d3-a0df-413e-abcb-f87ffc795c7d/csi-liveness-probe/0.log"