Jul 28 13:47:24.726005 ip-10-0-141-63 systemd[1]: Starting Kubernetes Kubelet... Jul 28 13:47:25.179411 ip-10-0-141-63 kubenswrapper[2550]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 13:47:25.179411 ip-10-0-141-63 kubenswrapper[2550]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 28 13:47:25.179411 ip-10-0-141-63 kubenswrapper[2550]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 13:47:25.179411 ip-10-0-141-63 kubenswrapper[2550]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 28 13:47:25.179411 ip-10-0-141-63 kubenswrapper[2550]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 13:47:25.180657 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:25.180567 2550 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 28 13:47:25.186421 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:25.186396 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 13:47:25.186421 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:25.186415 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 13:47:25.186421 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:25.186419 2550 feature_gate.go:328] unrecognized feature gate: Example Jul 28 13:47:25.186421 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:25.186422 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 13:47:25.186421 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:25.186425 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 13:47:25.186421 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:25.186429 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 13:47:25.186421 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:25.186432 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 13:47:26.194970 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:25.186435 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 13:47:26.194970 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:25.186438 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 13:47:26.194970 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:25.186441 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 13:47:26.194970 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:25.186443 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 13:47:26.194970 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:25.186446 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 13:47:26.194970 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:25.186449 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 13:47:26.194970 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:25.186451 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 13:47:26.194970 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:25.186454 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 13:47:26.194970 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:25.186456 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 13:47:26.194970 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:25.186459 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 13:47:26.194970 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:25.186462 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 13:47:26.194970 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:25.186464 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 13:47:26.194970 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:25.186467 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 13:47:26.194970 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:25.186470 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 13:47:26.194970 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:25.186472 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 13:47:26.194970 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194773 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 13:47:26.194970 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194775 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 13:47:26.194970 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194779 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 13:47:26.194970 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194782 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 13:47:26.195776 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194784 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 13:47:26.195776 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194788 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 13:47:26.195776 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194790 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 13:47:26.195776 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194792 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 13:47:26.195776 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194795 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 13:47:26.195776 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194799 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 13:47:26.195776 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194804 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 13:47:26.195776 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194807 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 13:47:26.195776 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194810 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 13:47:26.195776 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194812 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 13:47:26.195776 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194815 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 13:47:26.195776 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194818 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 13:47:26.195776 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194822 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 13:47:26.195776 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194825 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 13:47:26.195776 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194830 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 13:47:26.195776 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194833 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 13:47:26.195776 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194836 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 13:47:26.195776 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194839 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 13:47:26.195776 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194842 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 13:47:26.196286 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194845 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 13:47:26.196286 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194847 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 13:47:26.196286 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194850 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 13:47:26.196286 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194853 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 13:47:26.196286 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194856 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 13:47:26.196286 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194859 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 13:47:26.196286 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194863 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 13:47:26.196286 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194867 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 13:47:26.196286 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194870 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 13:47:26.196286 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194873 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 13:47:26.196286 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194875 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 13:47:26.196286 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194878 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 13:47:26.196286 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194881 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 13:47:26.196286 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194883 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 13:47:26.196286 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194886 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 13:47:26.196286 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194889 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 13:47:26.196286 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194891 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 13:47:26.196286 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194894 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 13:47:26.196286 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194896 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 13:47:26.196835 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194899 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 13:47:26.196835 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194901 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 13:47:26.196835 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194904 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 13:47:26.196835 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194907 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 13:47:26.196835 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194909 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 13:47:26.196835 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194912 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 13:47:26.196835 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194915 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 13:47:26.196835 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194917 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 13:47:26.196835 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194920 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 13:47:26.196835 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194924 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 13:47:26.196835 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194927 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 13:47:26.196835 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194930 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 13:47:26.196835 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194933 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 13:47:26.196835 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194935 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 13:47:26.196835 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194938 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 13:47:26.196835 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194940 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 13:47:26.196835 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194943 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 13:47:26.196835 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194945 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 13:47:26.196835 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194948 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 13:47:26.196835 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194951 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 13:47:26.197312 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194954 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 13:47:26.197312 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194956 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 13:47:26.197312 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194958 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 13:47:26.197312 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194961 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 13:47:26.197312 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194963 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 13:47:26.197312 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194966 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 13:47:26.197312 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194968 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 13:47:26.197312 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194971 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 13:47:26.197312 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194973 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 13:47:26.197312 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194976 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 13:47:26.197312 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194980 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 13:47:26.197312 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194982 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 13:47:26.197312 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194985 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 13:47:26.197312 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194988 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 13:47:26.197312 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.194991 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 13:47:26.197721 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197322 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 13:47:26.197721 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197330 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 13:47:26.197721 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197334 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 13:47:26.197721 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197337 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 13:47:26.197721 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197340 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 13:47:26.197721 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197343 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 13:47:26.197721 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197346 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 13:47:26.197721 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197348 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 13:47:26.197721 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197351 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 13:47:26.197721 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197355 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 13:47:26.197721 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197358 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 13:47:26.197721 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197360 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 13:47:26.197721 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197363 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 13:47:26.197721 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197365 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 13:47:26.197721 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197368 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 13:47:26.197721 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197370 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 13:47:26.197721 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197373 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 13:47:26.197721 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197376 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 13:47:26.197721 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197379 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 13:47:26.197721 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197381 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 13:47:26.198201 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197384 2550 feature_gate.go:328] unrecognized feature gate: Example Jul 28 13:47:26.198201 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197386 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 13:47:26.198201 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197403 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 13:47:26.198201 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197406 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 13:47:26.198201 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197409 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 13:47:26.198201 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197412 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 13:47:26.198201 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197415 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 13:47:26.198201 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197417 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 13:47:26.198201 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197420 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 13:47:26.198201 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197423 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 13:47:26.198201 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197425 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 13:47:26.198201 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197428 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 13:47:26.198201 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197431 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 13:47:26.198201 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197434 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 13:47:26.198201 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197436 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 13:47:26.198201 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197439 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 13:47:26.198201 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197442 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 13:47:26.198201 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197444 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 13:47:26.198201 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197447 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 13:47:26.198201 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197449 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 13:47:26.198756 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197454 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 13:47:26.198756 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197457 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 13:47:26.198756 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197460 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 13:47:26.198756 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197463 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 13:47:26.198756 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197466 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 13:47:26.198756 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197469 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 13:47:26.198756 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197471 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 13:47:26.198756 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197474 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 13:47:26.198756 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197476 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 13:47:26.198756 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197479 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 13:47:26.198756 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197481 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 13:47:26.198756 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197484 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 13:47:26.198756 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197486 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 13:47:26.198756 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197490 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 13:47:26.198756 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197494 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 13:47:26.198756 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197497 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 13:47:26.198756 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197499 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 13:47:26.198756 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197502 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 13:47:26.198756 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197504 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 13:47:26.199231 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197507 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 13:47:26.199231 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197510 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 13:47:26.199231 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197515 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 13:47:26.199231 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197520 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 13:47:26.199231 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197524 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 13:47:26.199231 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197527 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 13:47:26.199231 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197532 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 13:47:26.199231 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197535 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 13:47:26.199231 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197538 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 13:47:26.199231 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197541 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 13:47:26.199231 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197544 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 13:47:26.199231 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197547 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 13:47:26.199231 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197550 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 13:47:26.199231 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197552 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 13:47:26.199231 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197555 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 13:47:26.199231 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197557 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 13:47:26.199231 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197561 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 13:47:26.199231 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197565 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 13:47:26.199231 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197567 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 13:47:26.199705 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197570 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 13:47:26.199705 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197573 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 13:47:26.199705 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197576 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 13:47:26.199705 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197578 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 13:47:26.199705 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197581 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 13:47:26.199705 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197583 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 13:47:26.199705 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197586 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 13:47:26.199705 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197589 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 13:47:26.199705 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197591 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 13:47:26.199705 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197594 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 13:47:26.199705 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197597 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 13:47:26.199705 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197599 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 13:47:26.199705 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197602 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 13:47:26.199705 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197605 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 13:47:26.199705 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197607 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 13:47:26.199705 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197611 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 13:47:26.199705 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197614 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 13:47:26.199705 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197617 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 13:47:26.199705 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197619 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 13:47:26.199705 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197622 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 13:47:26.200196 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.197625 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 13:47:26.200196 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198004 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 13:47:26.200196 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198009 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 13:47:26.200196 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198012 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 13:47:26.200196 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198014 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 13:47:26.200196 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198017 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 13:47:26.200196 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198019 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 13:47:26.200196 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198022 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 13:47:26.200196 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198025 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 13:47:26.200196 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198028 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 13:47:26.200196 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198031 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 13:47:26.200196 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198033 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 13:47:26.200196 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198036 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 13:47:26.200196 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198038 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 13:47:26.200196 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198041 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 13:47:26.200196 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198043 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 13:47:26.200196 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198046 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 13:47:26.200196 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198048 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 13:47:26.200196 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198051 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 13:47:26.200677 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198054 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 13:47:26.200677 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198057 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 13:47:26.200677 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198059 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 13:47:26.200677 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198062 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 13:47:26.200677 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198064 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 13:47:26.200677 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198067 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 13:47:26.200677 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198070 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 13:47:26.200677 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198073 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 13:47:26.200677 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198076 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 13:47:26.200677 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198078 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 13:47:26.200677 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198081 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 13:47:26.200677 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198083 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 13:47:26.200677 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198086 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 13:47:26.200677 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198089 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 13:47:26.200677 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198091 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 13:47:26.200677 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198094 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 13:47:26.200677 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198096 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 13:47:26.200677 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198099 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 13:47:26.200677 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198101 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 13:47:26.200677 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198104 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 13:47:26.201184 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198106 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 13:47:26.201184 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198108 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 13:47:26.201184 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198111 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 13:47:26.201184 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198114 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 13:47:26.201184 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198117 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 13:47:26.201184 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198119 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 13:47:26.201184 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198122 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 13:47:26.201184 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198124 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 13:47:26.201184 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198127 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 13:47:26.201184 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198129 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 13:47:26.201184 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198133 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 13:47:26.201184 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198135 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 13:47:26.201184 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198138 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 13:47:26.201184 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198141 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 13:47:26.201184 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198143 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 13:47:26.201184 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198148 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 13:47:26.201184 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198151 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 13:47:26.201184 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198154 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 13:47:26.201184 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198157 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 13:47:26.201680 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198160 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 13:47:26.201680 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198163 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 13:47:26.201680 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198165 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 13:47:26.201680 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198168 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 13:47:26.201680 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198170 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 13:47:26.201680 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198175 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 13:47:26.201680 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198178 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 13:47:26.201680 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198182 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 13:47:26.201680 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198185 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 13:47:26.201680 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198187 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 13:47:26.201680 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198190 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 13:47:26.201680 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198193 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 13:47:26.201680 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198196 2550 feature_gate.go:328] unrecognized feature gate: Example Jul 28 13:47:26.201680 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198199 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 13:47:26.201680 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198202 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 13:47:26.201680 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198205 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 13:47:26.201680 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198208 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 13:47:26.201680 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198211 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 13:47:26.201680 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198214 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 13:47:26.201680 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198217 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 13:47:26.202154 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198220 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 13:47:26.202154 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198222 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 13:47:26.202154 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198228 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 13:47:26.202154 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198231 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 13:47:26.202154 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198233 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 13:47:26.202154 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198236 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 13:47:26.202154 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198238 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 13:47:26.202154 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198241 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 13:47:26.202154 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198243 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 13:47:26.202154 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198246 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 13:47:26.202154 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198248 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 13:47:26.202154 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198251 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 13:47:26.202154 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198253 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 13:47:26.202154 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198256 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 13:47:26.202154 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198258 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 13:47:26.202154 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198261 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 13:47:26.202154 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198263 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 13:47:26.202154 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198266 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 13:47:26.202154 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198268 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 13:47:26.202154 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198271 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 13:47:26.202722 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198273 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 13:47:26.202722 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.198276 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 13:47:26.202722 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198356 2550 flags.go:64] FLAG: --address="0.0.0.0" Jul 28 13:47:26.202722 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198368 2550 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 28 13:47:26.202722 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198401 2550 flags.go:64] FLAG: --anonymous-auth="true" Jul 28 13:47:26.202722 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198408 2550 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 28 13:47:26.202722 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198414 2550 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 28 13:47:26.202722 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198418 2550 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 28 13:47:26.202722 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198422 2550 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 28 13:47:26.202722 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198427 2550 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 28 13:47:26.202722 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198431 2550 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 28 13:47:26.202722 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198434 2550 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 28 13:47:26.202722 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198437 2550 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 28 13:47:26.202722 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198441 2550 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 28 13:47:26.202722 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198444 2550 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 28 13:47:26.202722 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198447 2550 flags.go:64] FLAG: --cgroup-root="" Jul 28 13:47:26.202722 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198450 2550 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 28 13:47:26.202722 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198453 2550 flags.go:64] FLAG: --client-ca-file="" Jul 28 13:47:26.202722 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198456 2550 flags.go:64] FLAG: --cloud-provider="external" Jul 28 13:47:26.202722 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198459 2550 flags.go:64] FLAG: --cluster-dns="[]" Jul 28 13:47:26.202722 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198464 2550 flags.go:64] FLAG: --cluster-domain="" Jul 28 13:47:26.202722 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198467 2550 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 28 13:47:26.202722 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198472 2550 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 28 13:47:26.202722 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198475 2550 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 28 13:47:26.203310 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198479 2550 flags.go:64] FLAG: --container-log-max-files="5" Jul 28 13:47:26.203310 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198483 2550 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 28 13:47:26.203310 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198486 2550 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 28 13:47:26.203310 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198489 2550 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 28 13:47:26.203310 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198492 2550 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 28 13:47:26.203310 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198495 2550 flags.go:64] FLAG: --contention-profiling="false" Jul 28 13:47:26.203310 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198498 2550 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 28 13:47:26.203310 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198501 2550 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 28 13:47:26.203310 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198504 2550 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 28 13:47:26.203310 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198507 2550 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 28 13:47:26.203310 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198511 2550 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 28 13:47:26.203310 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198514 2550 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 28 13:47:26.203310 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198517 2550 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 28 13:47:26.203310 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198520 2550 flags.go:64] FLAG: --enable-load-reader="false" Jul 28 13:47:26.203310 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198523 2550 flags.go:64] FLAG: --enable-server="true" Jul 28 13:47:26.203310 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198526 2550 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 28 13:47:26.203310 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198531 2550 flags.go:64] FLAG: --event-burst="100" Jul 28 13:47:26.203310 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198534 2550 flags.go:64] FLAG: --event-qps="50" Jul 28 13:47:26.203310 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198537 2550 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 28 13:47:26.203310 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198540 2550 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 28 13:47:26.203310 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198543 2550 flags.go:64] FLAG: --eviction-hard="" Jul 28 13:47:26.203310 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198557 2550 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 28 13:47:26.203310 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198560 2550 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 28 13:47:26.203310 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198565 2550 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 28 13:47:26.203310 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198568 2550 flags.go:64] FLAG: --eviction-soft="" Jul 28 13:47:26.204035 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198571 2550 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 28 13:47:26.204035 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198574 2550 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 28 13:47:26.204035 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198577 2550 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 28 13:47:26.204035 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198580 2550 flags.go:64] FLAG: --experimental-mounter-path="" Jul 28 13:47:26.204035 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198583 2550 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 28 13:47:26.204035 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198589 2550 flags.go:64] FLAG: --fail-swap-on="true" Jul 28 13:47:26.204035 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198593 2550 flags.go:64] FLAG: --feature-gates="" Jul 28 13:47:26.204035 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198597 2550 flags.go:64] FLAG: --file-check-frequency="20s" Jul 28 13:47:26.204035 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198600 2550 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 28 13:47:26.204035 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198603 2550 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 28 13:47:26.204035 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198607 2550 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 28 13:47:26.204035 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198610 2550 flags.go:64] FLAG: --healthz-port="10248" Jul 28 13:47:26.204035 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198613 2550 flags.go:64] FLAG: --help="false" Jul 28 13:47:26.204035 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198616 2550 flags.go:64] FLAG: --hostname-override="ip-10-0-141-63.ec2.internal" Jul 28 13:47:26.204035 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198619 2550 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 28 13:47:26.204035 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198622 2550 flags.go:64] FLAG: --http-check-frequency="20s" Jul 28 13:47:26.204035 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198625 2550 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 28 13:47:26.204035 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198629 2550 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 28 13:47:26.204035 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198632 2550 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 28 13:47:26.204035 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198635 2550 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 28 13:47:26.204035 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198638 2550 flags.go:64] FLAG: --image-service-endpoint="" Jul 28 13:47:26.204035 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198641 2550 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 28 13:47:26.204035 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198644 2550 flags.go:64] FLAG: --kube-api-burst="100" Jul 28 13:47:26.204681 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198647 2550 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 28 13:47:26.204681 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198650 2550 flags.go:64] FLAG: --kube-api-qps="50" Jul 28 13:47:26.204681 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198653 2550 flags.go:64] FLAG: --kube-reserved="" Jul 28 13:47:26.204681 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198656 2550 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 28 13:47:26.204681 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198659 2550 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 28 13:47:26.204681 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198663 2550 flags.go:64] FLAG: --kubelet-cgroups="" Jul 28 13:47:26.204681 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198666 2550 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 28 13:47:26.204681 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198670 2550 flags.go:64] FLAG: --lock-file="" Jul 28 13:47:26.204681 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198673 2550 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 28 13:47:26.204681 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198676 2550 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 28 13:47:26.204681 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198679 2550 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 28 13:47:26.204681 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198685 2550 flags.go:64] FLAG: --log-json-split-stream="false" Jul 28 13:47:26.204681 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198688 2550 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 28 13:47:26.204681 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198691 2550 flags.go:64] FLAG: --log-text-split-stream="false" Jul 28 13:47:26.204681 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198698 2550 flags.go:64] FLAG: --logging-format="text" Jul 28 13:47:26.204681 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198701 2550 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 28 13:47:26.204681 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198705 2550 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 28 13:47:26.204681 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198708 2550 flags.go:64] FLAG: --manifest-url="" Jul 28 13:47:26.204681 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198711 2550 flags.go:64] FLAG: --manifest-url-header="" Jul 28 13:47:26.204681 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198716 2550 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 28 13:47:26.204681 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198719 2550 flags.go:64] FLAG: --max-open-files="1000000" Jul 28 13:47:26.204681 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198724 2550 flags.go:64] FLAG: --max-pods="110" Jul 28 13:47:26.204681 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198727 2550 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 28 13:47:26.204681 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198730 2550 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 28 13:47:26.204681 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198733 2550 flags.go:64] FLAG: --memory-manager-policy="None" Jul 28 13:47:26.205302 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198736 2550 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 28 13:47:26.205302 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198739 2550 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 28 13:47:26.205302 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198742 2550 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 28 13:47:26.205302 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198745 2550 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 28 13:47:26.205302 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198753 2550 flags.go:64] FLAG: --node-status-max-images="50" Jul 28 13:47:26.205302 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198756 2550 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 28 13:47:26.205302 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198759 2550 flags.go:64] FLAG: --oom-score-adj="-999" Jul 28 13:47:26.205302 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198763 2550 flags.go:64] FLAG: --pod-cidr="" Jul 28 13:47:26.205302 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198766 2550 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 28 13:47:26.205302 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198771 2550 flags.go:64] FLAG: --pod-manifest-path="" Jul 28 13:47:26.205302 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198774 2550 flags.go:64] FLAG: --pod-max-pids="-1" Jul 28 13:47:26.205302 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198778 2550 flags.go:64] FLAG: --pods-per-core="0" Jul 28 13:47:26.205302 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198781 2550 flags.go:64] FLAG: --port="10250" Jul 28 13:47:26.205302 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198784 2550 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 28 13:47:26.205302 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198787 2550 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03dafaf5f070a64f3" Jul 28 13:47:26.205302 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198792 2550 flags.go:64] FLAG: --qos-reserved="" Jul 28 13:47:26.205302 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198795 2550 flags.go:64] FLAG: --read-only-port="10255" Jul 28 13:47:26.205302 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198798 2550 flags.go:64] FLAG: --register-node="true" Jul 28 13:47:26.205302 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198801 2550 flags.go:64] FLAG: --register-with-taints="" Jul 28 13:47:26.205302 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198804 2550 flags.go:64] FLAG: --registry-burst="10" Jul 28 13:47:26.205302 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198807 2550 flags.go:64] FLAG: --registry-qps="5" Jul 28 13:47:26.205302 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198811 2550 flags.go:64] FLAG: --reserved-cpus="" Jul 28 13:47:26.205302 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198828 2550 flags.go:64] FLAG: --reserved-memory="" Jul 28 13:47:26.205302 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198835 2550 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 28 13:47:26.205302 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198838 2550 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 28 13:47:26.205941 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198841 2550 flags.go:64] FLAG: --rotate-certificates="false" Jul 28 13:47:26.205941 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198844 2550 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 28 13:47:26.205941 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198847 2550 flags.go:64] FLAG: --runonce="false" Jul 28 13:47:26.205941 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198850 2550 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 28 13:47:26.205941 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198853 2550 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 28 13:47:26.205941 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198856 2550 flags.go:64] FLAG: --seccomp-default="false" Jul 28 13:47:26.205941 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198859 2550 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 28 13:47:26.205941 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198862 2550 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 28 13:47:26.205941 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198865 2550 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 28 13:47:26.205941 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198868 2550 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 28 13:47:26.205941 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198872 2550 flags.go:64] FLAG: --storage-driver-password="root" Jul 28 13:47:26.205941 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198875 2550 flags.go:64] FLAG: --storage-driver-secure="false" Jul 28 13:47:26.205941 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198878 2550 flags.go:64] FLAG: --storage-driver-table="stats" Jul 28 13:47:26.205941 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198880 2550 flags.go:64] FLAG: --storage-driver-user="root" Jul 28 13:47:26.205941 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198883 2550 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 28 13:47:26.205941 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198887 2550 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 28 13:47:26.205941 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198890 2550 flags.go:64] FLAG: --system-cgroups="" Jul 28 13:47:26.205941 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198893 2550 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 28 13:47:26.205941 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198898 2550 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 28 13:47:26.205941 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198901 2550 flags.go:64] FLAG: --tls-cert-file="" Jul 28 13:47:26.205941 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198905 2550 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 28 13:47:26.205941 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198910 2550 flags.go:64] FLAG: --tls-min-version="" Jul 28 13:47:26.205941 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198914 2550 flags.go:64] FLAG: --tls-private-key-file="" Jul 28 13:47:26.205941 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198917 2550 flags.go:64] FLAG: --topology-manager-policy="none" Jul 28 13:47:26.205941 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198920 2550 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 28 13:47:26.206625 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198923 2550 flags.go:64] FLAG: --topology-manager-scope="container" Jul 28 13:47:26.206625 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198926 2550 flags.go:64] FLAG: --v="2" Jul 28 13:47:26.206625 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198930 2550 flags.go:64] FLAG: --version="false" Jul 28 13:47:26.206625 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198935 2550 flags.go:64] FLAG: --vmodule="" Jul 28 13:47:26.206625 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198939 2550 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 28 13:47:26.206625 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.198942 2550 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 28 13:47:26.206625 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199077 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 13:47:26.206625 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199082 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 13:47:26.206625 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199085 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 13:47:26.206625 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199088 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 13:47:26.206625 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199091 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 13:47:26.206625 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199093 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 13:47:26.206625 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199096 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 13:47:26.206625 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199099 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 13:47:26.206625 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199102 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 13:47:26.206625 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199105 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 13:47:26.206625 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199107 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 13:47:26.206625 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199110 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 13:47:26.206625 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199113 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 13:47:26.206625 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199115 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 13:47:26.206625 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199118 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 13:47:26.207156 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199120 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 13:47:26.207156 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199123 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 13:47:26.207156 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199126 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 13:47:26.207156 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199129 2550 feature_gate.go:328] unrecognized feature gate: Example Jul 28 13:47:26.207156 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199132 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 13:47:26.207156 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199135 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 13:47:26.207156 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199137 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 13:47:26.207156 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199141 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 13:47:26.207156 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199145 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 13:47:26.207156 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199148 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 13:47:26.207156 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199151 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 13:47:26.207156 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199155 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 13:47:26.207156 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199159 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 13:47:26.207156 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199162 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 13:47:26.207156 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199166 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 13:47:26.207156 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199169 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 13:47:26.207156 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199172 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 13:47:26.207156 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199174 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 13:47:26.207156 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199177 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 13:47:26.207156 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199180 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 13:47:26.207675 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199182 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 13:47:26.207675 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199186 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 13:47:26.207675 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199190 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 13:47:26.207675 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199193 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 13:47:26.207675 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199195 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 13:47:26.207675 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199198 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 13:47:26.207675 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199201 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 13:47:26.207675 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199203 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 13:47:26.207675 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199206 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 13:47:26.207675 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199209 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 13:47:26.207675 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199212 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 13:47:26.207675 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199214 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 13:47:26.207675 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199217 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 13:47:26.207675 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199219 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 13:47:26.207675 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199222 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 13:47:26.207675 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199225 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 13:47:26.207675 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199229 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 13:47:26.207675 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199232 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 13:47:26.207675 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199236 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 13:47:26.208177 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199239 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 13:47:26.208177 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199244 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 13:47:26.208177 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199247 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 13:47:26.208177 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199250 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 13:47:26.208177 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199253 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 13:47:26.208177 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199256 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 13:47:26.208177 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199259 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 13:47:26.208177 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199263 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 13:47:26.208177 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199265 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 13:47:26.208177 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199268 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 13:47:26.208177 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199271 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 13:47:26.208177 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199273 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 13:47:26.208177 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199276 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 13:47:26.208177 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199279 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 13:47:26.208177 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199281 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 13:47:26.208177 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199284 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 13:47:26.208177 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199287 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 13:47:26.208177 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199289 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 13:47:26.208177 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199292 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 13:47:26.208667 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199295 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 13:47:26.208667 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199297 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 13:47:26.208667 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199300 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 13:47:26.208667 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199302 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 13:47:26.208667 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199305 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 13:47:26.208667 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199307 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 13:47:26.208667 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199310 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 13:47:26.208667 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199313 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 13:47:26.208667 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199315 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 13:47:26.208667 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199318 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 13:47:26.208667 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199320 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 13:47:26.208667 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199323 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 13:47:26.208667 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199325 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 13:47:26.208667 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199328 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 13:47:26.208667 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199331 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 13:47:26.208667 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199334 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 13:47:26.208667 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199337 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 13:47:26.208667 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199340 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 13:47:26.208667 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199342 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 13:47:26.208667 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199345 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 13:47:26.209169 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199349 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 13:47:26.209169 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199351 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 13:47:26.209169 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199354 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 13:47:26.209169 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199357 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 13:47:26.209169 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199359 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 13:47:26.209169 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.199362 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 13:47:26.209169 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.200164 2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 13:47:26.209518 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.209497 2550 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 28 13:47:26.209556 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.209519 2550 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 28 13:47:26.209556 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.209537 2550 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 28 13:47:26.209556 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.209543 2550 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 28 13:47:26.209636 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209600 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 13:47:26.209636 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209606 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 13:47:26.209636 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209610 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 13:47:26.209636 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209613 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 13:47:26.209636 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209616 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 13:47:26.209636 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209619 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 13:47:26.209636 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209622 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 13:47:26.209636 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209625 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 13:47:26.209636 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209627 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 13:47:26.209636 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209630 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 13:47:26.209636 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209632 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 13:47:26.209636 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209635 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 13:47:26.209636 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209638 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 13:47:26.209636 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209641 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 13:47:26.209636 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209644 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 13:47:26.210010 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209646 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 13:47:26.210010 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209649 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 13:47:26.210010 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209652 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 13:47:26.210010 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209654 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 13:47:26.210010 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209657 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 13:47:26.210010 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209660 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 13:47:26.210010 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209662 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 13:47:26.210010 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209665 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 13:47:26.210010 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209667 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 13:47:26.210010 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209670 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 13:47:26.210010 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209672 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 13:47:26.210010 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209675 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 13:47:26.210010 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209678 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 13:47:26.210010 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209680 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 13:47:26.210010 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209683 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 13:47:26.210010 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209686 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 13:47:26.210010 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209689 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 13:47:26.210010 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209693 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 13:47:26.210010 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209697 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 13:47:26.210010 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209700 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 13:47:26.210577 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209702 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 13:47:26.210577 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209705 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 13:47:26.210577 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209707 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 13:47:26.210577 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209710 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 13:47:26.210577 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209712 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 13:47:26.210577 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209715 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 13:47:26.210577 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209717 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 13:47:26.210577 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209720 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 13:47:26.210577 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209722 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 13:47:26.210577 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209725 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 13:47:26.210577 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209727 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 13:47:26.210577 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209730 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 13:47:26.210577 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209734 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 13:47:26.210577 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209737 2550 feature_gate.go:328] unrecognized feature gate: Example Jul 28 13:47:26.210577 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209739 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 13:47:26.210577 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209742 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 13:47:26.210577 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209745 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 13:47:26.210577 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209748 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 13:47:26.210577 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209750 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 13:47:26.210577 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209753 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 13:47:26.211087 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209756 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 13:47:26.211087 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209758 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 13:47:26.211087 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209761 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 13:47:26.211087 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209763 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 13:47:26.211087 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209766 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 13:47:26.211087 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209768 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 13:47:26.211087 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209771 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 13:47:26.211087 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209774 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 13:47:26.211087 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209777 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 13:47:26.211087 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209779 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 13:47:26.211087 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209782 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 13:47:26.211087 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209785 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 13:47:26.211087 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209787 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 13:47:26.211087 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209790 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 13:47:26.211087 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209793 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 13:47:26.211087 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209795 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 13:47:26.211087 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209798 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 13:47:26.211087 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209801 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 13:47:26.211087 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209804 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 13:47:26.211591 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209808 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 13:47:26.211591 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209811 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 13:47:26.211591 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209814 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 13:47:26.211591 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209817 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 13:47:26.211591 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209820 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 13:47:26.211591 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209823 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 13:47:26.211591 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209826 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 13:47:26.211591 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209829 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 13:47:26.211591 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209832 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 13:47:26.211591 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209835 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 13:47:26.211591 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209838 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 13:47:26.211591 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209841 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 13:47:26.211591 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209844 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 13:47:26.211591 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209847 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 13:47:26.211591 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209849 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 13:47:26.211591 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209852 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 13:47:26.211591 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209855 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 13:47:26.211591 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209857 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 13:47:26.211591 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209860 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 13:47:26.212059 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209863 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 13:47:26.212059 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209865 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 13:47:26.212059 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209868 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 13:47:26.212059 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209871 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 13:47:26.212059 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209873 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 13:47:26.212059 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209876 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 13:47:26.212059 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.209881 2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 13:47:26.212059 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209993 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 13:47:26.212059 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.209999 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 13:47:26.212059 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210002 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 13:47:26.212059 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210005 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 13:47:26.212059 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210007 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 13:47:26.212059 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210011 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 13:47:26.212059 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210013 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 13:47:26.212059 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210016 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 13:47:26.212456 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210019 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 13:47:26.212456 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210022 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 13:47:26.212456 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210025 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 13:47:26.212456 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210029 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 13:47:26.212456 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210032 2550 feature_gate.go:328] unrecognized feature gate: Example Jul 28 13:47:26.212456 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210035 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 13:47:26.212456 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210037 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 13:47:26.212456 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210040 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 13:47:26.212456 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210043 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 13:47:26.212456 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210045 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 13:47:26.212456 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210048 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 13:47:26.212456 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210050 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 13:47:26.212456 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210053 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 13:47:26.212456 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210055 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 13:47:26.212456 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210058 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 13:47:26.212456 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210060 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 13:47:26.212456 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210063 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 13:47:26.212456 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210065 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 13:47:26.212456 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210068 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 13:47:26.212456 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210071 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 13:47:26.213000 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210074 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 13:47:26.213000 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210077 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 13:47:26.213000 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210079 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 13:47:26.213000 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210082 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 13:47:26.213000 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210084 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 13:47:26.213000 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210087 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 13:47:26.213000 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210089 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 13:47:26.213000 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210092 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 13:47:26.213000 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210094 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 13:47:26.213000 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210097 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 13:47:26.213000 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210099 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 13:47:26.213000 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210102 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 13:47:26.213000 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210104 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 13:47:26.213000 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210107 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 13:47:26.213000 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210110 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 13:47:26.213000 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210113 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 13:47:26.213000 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210115 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 13:47:26.213000 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210118 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 13:47:26.213000 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210120 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 13:47:26.213000 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210123 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 13:47:26.213807 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210125 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 13:47:26.213807 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210128 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 13:47:26.213807 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210130 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 13:47:26.213807 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210133 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 13:47:26.213807 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210135 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 13:47:26.213807 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210138 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 13:47:26.213807 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210140 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 13:47:26.213807 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210142 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 13:47:26.213807 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210145 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 13:47:26.213807 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210147 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 13:47:26.213807 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210150 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 13:47:26.213807 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210153 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 13:47:26.213807 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210156 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 13:47:26.213807 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210158 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 13:47:26.213807 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210162 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 13:47:26.213807 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210165 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 13:47:26.213807 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210168 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 13:47:26.213807 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210171 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 13:47:26.213807 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210173 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 13:47:26.213807 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210176 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 13:47:26.214462 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210179 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 13:47:26.214462 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210182 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 13:47:26.214462 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210184 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 13:47:26.214462 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210187 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 13:47:26.214462 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210189 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 13:47:26.214462 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210192 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 13:47:26.214462 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210195 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 13:47:26.214462 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210197 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 13:47:26.214462 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210200 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 13:47:26.214462 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210202 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 13:47:26.214462 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210205 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 13:47:26.214462 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210207 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 13:47:26.214462 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210210 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 13:47:26.214462 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210212 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 13:47:26.214462 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210215 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 13:47:26.214462 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210218 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 13:47:26.214462 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210220 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 13:47:26.214462 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210222 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 13:47:26.214462 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210225 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 13:47:26.214951 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210227 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 13:47:26.214951 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210230 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 13:47:26.214951 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210232 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 13:47:26.214951 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210235 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 13:47:26.214951 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210237 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 13:47:26.214951 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210240 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 13:47:26.214951 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210243 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 13:47:26.214951 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210246 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 13:47:26.214951 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210249 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 13:47:26.214951 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210253 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 13:47:26.214951 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210256 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 13:47:26.214951 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:26.210259 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 13:47:26.214951 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.210264 2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 13:47:26.214951 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.210422 2550 server.go:962] "Client rotation is on, will bootstrap in background" Jul 28 13:47:26.215296 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.212597 2550 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 28 13:47:26.216614 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.216596 2550 server.go:1019] "Starting client certificate rotation" Jul 28 13:47:26.216729 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.216709 2550 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 28 13:47:26.216774 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.216762 2550 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 28 13:47:26.219645 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.219629 2550 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 28 13:47:26.220860 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.220841 2550 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 28 13:47:26.232051 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.232031 2550 log.go:25] "Validated CRI v1 runtime API" Jul 28 13:47:26.234297 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.234277 2550 log.go:25] "Validated CRI v1 image API" Jul 28 13:47:26.235646 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.235631 2550 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 28 13:47:26.238143 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.238111 2550 fs.go:135] Filesystem UUIDs: map[6739f9ef-ef1e-4073-98bb-b13eb3e1c1d7:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 7d307930-e214-4286-afdd-84f9b53f78d9:/dev/nvme0n1p3] Jul 28 13:47:26.238233 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.238138 2550 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 28 13:47:26.244648 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.244132 2550 manager.go:216] Machine: {Timestamp:2026-07-28 13:47:26.242966396 +0000 UTC m=+0.378163279 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099651 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2eb32c0158656713cc3606dd48c4ba SystemUUID:ec2eb32c-0158-6567-13cc-3606dd48c4ba BootID:ba80767f-b7b2-4868-b3e8-b629c7e6e1e6 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:25:6d:31:74:c1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:25:6d:31:74:c1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e6:8f:52:c4:a7:dd Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 28 13:47:26.244648 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.244633 2550 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 28 13:47:26.244782 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.244719 2550 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 28 13:47:26.246029 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.246009 2550 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 28 13:47:26.247706 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.247673 2550 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 28 13:47:26.247835 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.247706 2550 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-63.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 28 13:47:26.248967 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.248956 2550 topology_manager.go:138] "Creating topology manager with none policy" Jul 28 13:47:26.249013 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.248970 2550 container_manager_linux.go:309] "Creating device plugin manager" Jul 28 13:47:26.249013 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.248978 2550 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 28 13:47:26.249607 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.249597 2550 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 28 13:47:26.250426 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.250416 2550 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 28 13:47:26.252364 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.252353 2550 state_mem.go:36] "Initialized new in-memory state store" Jul 28 13:47:26.252691 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.252680 2550 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 28 13:47:26.252729 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.252713 2550 kubelet.go:486] "Attempting to sync node with API server" Jul 28 13:47:26.252729 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.252728 2550 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 28 13:47:26.252789 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.252739 2550 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 28 13:47:26.252789 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.252749 2550 kubelet.go:398] "Adding apiserver pod source" Jul 28 13:47:26.252789 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.252757 2550 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 28 13:47:26.253192 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.253171 2550 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 28 13:47:26.253227 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.253195 2550 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 28 13:47:26.255373 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.255356 2550 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 28 13:47:26.257333 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.257320 2550 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 28 13:47:26.257377 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.257351 2550 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 28 13:47:26.259059 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.259047 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 28 13:47:26.259059 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.259063 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 28 13:47:26.259164 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.259069 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 28 13:47:26.259164 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.259074 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 28 13:47:26.259164 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.259084 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 28 13:47:26.259164 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.259090 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 28 13:47:26.259164 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.259098 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 28 13:47:26.259164 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.259103 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 28 13:47:26.259164 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.259120 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 28 13:47:26.259164 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.259126 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 28 13:47:26.259164 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.259141 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 28 13:47:26.259164 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.259146 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 28 13:47:26.261269 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.261253 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 28 13:47:26.261333 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.261278 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 28 13:47:26.265800 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.265781 2550 server.go:1268] "Started kubelet" Jul 28 13:47:26.265985 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.265925 2550 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 28 13:47:26.266084 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.266011 2550 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 28 13:47:26.266231 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.266201 2550 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-63.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 28 13:47:26.266306 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:26.266290 2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 28 13:47:26.266434 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:26.266414 2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-63.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 28 13:47:26.266525 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.266501 2550 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 28 13:47:26.266866 ip-10-0-141-63 systemd[1]: Started Kubernetes Kubelet. Jul 28 13:47:26.267602 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.267584 2550 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 28 13:47:26.270745 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.270729 2550 server.go:310] "Adding debug handlers to kubelet server" Jul 28 13:47:26.271378 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:26.271355 2550 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 28 13:47:26.272178 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:26.271311 2550 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-63.ec2.internal.18c6782f2a32dbd8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-63.ec2.internal,UID:ip-10-0-141-63.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-63.ec2.internal,},FirstTimestamp:2026-07-28 13:47:26.265752536 +0000 UTC m=+0.400949419,LastTimestamp:2026-07-28 13:47:26.265752536 +0000 UTC m=+0.400949419,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-63.ec2.internal,}" Jul 28 13:47:26.274312 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.274295 2550 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 28 13:47:26.274428 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.274314 2550 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 28 13:47:26.274428 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:26.274407 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-63.ec2.internal\" not found" Jul 28 13:47:26.275067 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.275049 2550 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 28 13:47:26.275265 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.275235 2550 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 28 13:47:26.275265 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.275265 2550 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 28 13:47:26.275383 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.275326 2550 factory.go:153] Registering CRI-O factory Jul 28 13:47:26.275383 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.275341 2550 factory.go:223] Registration of the crio container factory successfully Jul 28 13:47:26.275383 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.275374 2550 reconstruct.go:97] "Volume reconstruction finished" Jul 28 13:47:26.275383 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.275382 2550 reconciler.go:29] "Reconciler: start to sync state" Jul 28 13:47:26.275383 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.275406 2550 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 28 13:47:26.275383 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.275417 2550 factory.go:55] Registering systemd factory Jul 28 13:47:26.275383 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.275425 2550 factory.go:223] Registration of the systemd container factory successfully Jul 28 13:47:26.275827 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.275449 2550 factory.go:103] Registering Raw factory Jul 28 13:47:26.275827 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.275464 2550 manager.go:1196] Started watching for new ooms in manager Jul 28 13:47:26.276044 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.275839 2550 manager.go:349] Starting recovery of all containers Jul 28 13:47:26.283076 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.282913 2550 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mr5c5" Jul 28 13:47:26.283218 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.283190 2550 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 28 13:47:26.284999 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:26.284965 2550 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-63.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 28 13:47:26.285108 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:26.285081 2550 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 28 13:47:26.285462 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.285375 2550 manager.go:354] Recovery completed Jul 28 13:47:26.288071 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:26.288032 2550 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 28 13:47:26.292109 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.292095 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 13:47:26.293985 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.293966 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-63.ec2.internal" event="NodeHasSufficientMemory" Jul 28 13:47:26.294063 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.294004 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-63.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 13:47:26.294063 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.294016 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-63.ec2.internal" event="NodeHasSufficientPID" Jul 28 13:47:26.294612 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.294598 2550 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 28 13:47:26.294612 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.294611 2550 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 28 13:47:26.294697 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.294622 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 13:47:26.294697 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.294630 2550 state_mem.go:36] "Initialized new in-memory state store" Jul 28 13:47:26.295295 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.295280 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-63.ec2.internal" event="NodeHasSufficientMemory" Jul 28 13:47:26.295355 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.295308 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-63.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 13:47:26.295355 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.295322 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-63.ec2.internal" event="NodeHasSufficientPID" Jul 28 13:47:26.295445 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.295431 2550 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mr5c5" Jul 28 13:47:26.296110 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:26.296038 2550 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-63.ec2.internal.18c6782f2be1a0b9 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-63.ec2.internal,UID:ip-10-0-141-63.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-63.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-63.ec2.internal,},FirstTimestamp:2026-07-28 13:47:26.293983417 +0000 UTC m=+0.429180303,LastTimestamp:2026-07-28 13:47:26.293983417 +0000 UTC m=+0.429180303,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-63.ec2.internal,}" Jul 28 13:47:26.296985 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.296972 2550 policy_none.go:49] "None policy: Start" Jul 28 13:47:26.297037 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.296988 2550 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 28 13:47:26.297037 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.296999 2550 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 28 13:47:26.298162 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.298147 2550 policy_none.go:47] "Start" Jul 28 13:47:26.343082 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.343063 2550 manager.go:337] "Starting Device Plugin manager" Jul 28 13:47:26.343206 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:26.343100 2550 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 28 13:47:26.343206 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.343110 2550 server.go:87] "Starting device plugin registration server" Jul 28 13:47:26.343447 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.343433 2550 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 28 13:47:26.343499 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.343449 2550 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 28 13:47:26.343547 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.343527 2550 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 28 13:47:26.343619 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.343604 2550 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 28 13:47:26.343619 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.343617 2550 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 28 13:47:26.344213 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:26.344192 2550 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 28 13:47:26.344333 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:26.344243 2550 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-63.ec2.internal\" not found" Jul 28 13:47:26.443640 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.443570 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 13:47:26.444794 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.444744 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-63.ec2.internal" event="NodeHasSufficientMemory" Jul 28 13:47:26.444794 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.444780 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-63.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 13:47:26.444794 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.444790 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-63.ec2.internal" event="NodeHasSufficientPID" Jul 28 13:47:26.444932 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.444818 2550 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-63.ec2.internal" Jul 28 13:47:26.454410 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.454373 2550 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-63.ec2.internal" Jul 28 13:47:26.454536 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:26.454433 2550 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-63.ec2.internal\": node \"ip-10-0-141-63.ec2.internal\" not found" Jul 28 13:47:26.469228 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:26.469205 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-63.ec2.internal\" not found" Jul 28 13:47:26.471344 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.471325 2550 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 28 13:47:26.471435 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.471368 2550 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 28 13:47:26.471435 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.471387 2550 kubelet.go:2448] "Starting kubelet main sync loop" Jul 28 13:47:26.471517 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:26.471438 2550 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 28 13:47:26.473004 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.472987 2550 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 13:47:26.569359 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:26.569307 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-63.ec2.internal\" not found" Jul 28 13:47:26.572462 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.572432 2550 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-63.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-63.ec2.internal"] Jul 28 13:47:26.572588 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.572508 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 13:47:26.574102 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.574084 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-63.ec2.internal" event="NodeHasSufficientMemory" Jul 28 13:47:26.574204 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.574117 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-63.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 13:47:26.574204 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.574127 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-63.ec2.internal" event="NodeHasSufficientPID" Jul 28 13:47:26.575495 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.575482 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 13:47:26.575596 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.575581 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-63.ec2.internal" Jul 28 13:47:26.575643 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.575613 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 13:47:26.576214 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.576198 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-63.ec2.internal" event="NodeHasSufficientMemory" Jul 28 13:47:26.576307 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.576228 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-63.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 13:47:26.576307 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.576242 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-63.ec2.internal" event="NodeHasSufficientPID" Jul 28 13:47:26.576307 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.576198 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-63.ec2.internal" event="NodeHasSufficientMemory" Jul 28 13:47:26.576307 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.576300 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-63.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 13:47:26.576510 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.576313 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-63.ec2.internal" event="NodeHasSufficientPID" Jul 28 13:47:26.577438 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.577422 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-63.ec2.internal" Jul 28 13:47:26.577507 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.577448 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 13:47:26.577507 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.577483 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/021d5cd1abd768b313395dd23c6c4054-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-63.ec2.internal\" (UID: \"021d5cd1abd768b313395dd23c6c4054\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-63.ec2.internal" Jul 28 13:47:26.577590 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.577515 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/021d5cd1abd768b313395dd23c6c4054-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-63.ec2.internal\" (UID: \"021d5cd1abd768b313395dd23c6c4054\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-63.ec2.internal" Jul 28 13:47:26.578178 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.578161 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-63.ec2.internal" event="NodeHasSufficientMemory" Jul 28 13:47:26.578258 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.578185 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-63.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 13:47:26.578258 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.578197 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-63.ec2.internal" event="NodeHasSufficientPID" Jul 28 13:47:26.607055 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:26.607025 2550 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-63.ec2.internal\" not found" node="ip-10-0-141-63.ec2.internal" Jul 28 13:47:26.611691 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:26.611674 2550 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-63.ec2.internal\" not found" node="ip-10-0-141-63.ec2.internal" Jul 28 13:47:26.670457 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:26.670428 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-63.ec2.internal\" not found" Jul 28 13:47:26.677770 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.677750 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/021d5cd1abd768b313395dd23c6c4054-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-63.ec2.internal\" (UID: \"021d5cd1abd768b313395dd23c6c4054\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-63.ec2.internal" Jul 28 13:47:26.677839 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.677774 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/021d5cd1abd768b313395dd23c6c4054-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-63.ec2.internal\" (UID: \"021d5cd1abd768b313395dd23c6c4054\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-63.ec2.internal" Jul 28 13:47:26.677839 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.677795 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d99d1c9d97e7e4c15b53cfee9d40b63c-config\") pod \"kube-apiserver-proxy-ip-10-0-141-63.ec2.internal\" (UID: \"d99d1c9d97e7e4c15b53cfee9d40b63c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-63.ec2.internal" Jul 28 13:47:26.677914 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.677844 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/021d5cd1abd768b313395dd23c6c4054-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-63.ec2.internal\" (UID: \"021d5cd1abd768b313395dd23c6c4054\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-63.ec2.internal" Jul 28 13:47:26.677914 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.677862 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/021d5cd1abd768b313395dd23c6c4054-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-63.ec2.internal\" (UID: \"021d5cd1abd768b313395dd23c6c4054\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-63.ec2.internal" Jul 28 13:47:26.771255 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:26.771179 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-63.ec2.internal\" not found" Jul 28 13:47:26.778539 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.778518 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d99d1c9d97e7e4c15b53cfee9d40b63c-config\") pod \"kube-apiserver-proxy-ip-10-0-141-63.ec2.internal\" (UID: \"d99d1c9d97e7e4c15b53cfee9d40b63c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-63.ec2.internal" Jul 28 13:47:26.778539 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.778476 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d99d1c9d97e7e4c15b53cfee9d40b63c-config\") pod \"kube-apiserver-proxy-ip-10-0-141-63.ec2.internal\" (UID: \"d99d1c9d97e7e4c15b53cfee9d40b63c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-63.ec2.internal" Jul 28 13:47:26.871685 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:26.871649 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-63.ec2.internal\" not found" Jul 28 13:47:26.910198 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.910181 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-63.ec2.internal" Jul 28 13:47:26.915416 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:26.915401 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-63.ec2.internal" Jul 28 13:47:26.972463 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:26.972428 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-63.ec2.internal\" not found" Jul 28 13:47:27.072980 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:27.072898 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-63.ec2.internal\" not found" Jul 28 13:47:27.090560 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.090533 2550 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 13:47:27.175161 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.175139 2550 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-63.ec2.internal" Jul 28 13:47:27.190665 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.190640 2550 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 28 13:47:27.192476 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.192460 2550 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-63.ec2.internal" Jul 28 13:47:27.199780 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.199764 2550 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 28 13:47:27.213918 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.213898 2550 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 28 13:47:27.214059 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.214041 2550 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 28 13:47:27.214094 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.214068 2550 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 28 13:47:27.253234 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.253208 2550 apiserver.go:52] "Watching apiserver" Jul 28 13:47:27.272368 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.272342 2550 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 28 13:47:27.272717 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.272696 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-89dlx","kube-system/konnectivity-agent-vvwjc","kube-system/kube-apiserver-proxy-ip-10-0-141-63.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wktfs","openshift-cluster-node-tuning-operator/tuned-v2wxp","openshift-dns/node-resolver-bgkv9","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-63.ec2.internal","openshift-multus/multus-6d5r6","openshift-image-registry/node-ca-qvpwc","openshift-multus/multus-additional-cni-plugins-8br7q","openshift-multus/network-metrics-daemon-khglj","openshift-network-diagnostics/network-check-target-2z2sg","openshift-network-operator/iptables-alerter-j5v9s","openshift-ovn-kubernetes/ovnkube-node-4nb4j"] Jul 28 13:47:27.274483 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.274465 2550 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 28 13:47:27.274718 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.274700 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8br7q" Jul 28 13:47:27.277593 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.277571 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 28 13:47:27.277738 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.277716 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-j5v9s" Jul 28 13:47:27.277822 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.277757 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 28 13:47:27.277822 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.277722 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 28 13:47:27.278033 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.277939 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 28 13:47:27.278591 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.278575 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 28 13:47:27.278666 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.278650 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-f8rss\"" Jul 28 13:47:27.278739 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.278723 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.280105 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.280092 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wktfs" Jul 28 13:47:27.280964 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.280831 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 28 13:47:27.280964 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.280863 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 28 13:47:27.280964 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.280866 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-kq27l\"" Jul 28 13:47:27.280964 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.280917 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 28 13:47:27.280964 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.280927 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/51d01c0c-2588-432a-bce0-03459112ffb9-system-cni-dir\") pod \"multus-additional-cni-plugins-8br7q\" (UID: \"51d01c0c-2588-432a-bce0-03459112ffb9\") " pod="openshift-multus/multus-additional-cni-plugins-8br7q" Jul 28 13:47:27.280964 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.280958 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/51d01c0c-2588-432a-bce0-03459112ffb9-cni-binary-copy\") pod \"multus-additional-cni-plugins-8br7q\" (UID: \"51d01c0c-2588-432a-bce0-03459112ffb9\") " pod="openshift-multus/multus-additional-cni-plugins-8br7q" Jul 28 13:47:27.281267 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.280985 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/51d01c0c-2588-432a-bce0-03459112ffb9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8br7q\" (UID: \"51d01c0c-2588-432a-bce0-03459112ffb9\") " pod="openshift-multus/multus-additional-cni-plugins-8br7q" Jul 28 13:47:27.281267 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.281006 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/51d01c0c-2588-432a-bce0-03459112ffb9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8br7q\" (UID: \"51d01c0c-2588-432a-bce0-03459112ffb9\") " pod="openshift-multus/multus-additional-cni-plugins-8br7q" Jul 28 13:47:27.281267 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.281047 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kz9j9\" (UniqueName: \"kubernetes.io/projected/51d01c0c-2588-432a-bce0-03459112ffb9-kube-api-access-kz9j9\") pod \"multus-additional-cni-plugins-8br7q\" (UID: \"51d01c0c-2588-432a-bce0-03459112ffb9\") " pod="openshift-multus/multus-additional-cni-plugins-8br7q" Jul 28 13:47:27.281267 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.281080 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/51d01c0c-2588-432a-bce0-03459112ffb9-cnibin\") pod \"multus-additional-cni-plugins-8br7q\" (UID: \"51d01c0c-2588-432a-bce0-03459112ffb9\") " pod="openshift-multus/multus-additional-cni-plugins-8br7q" Jul 28 13:47:27.281267 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.281131 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/51d01c0c-2588-432a-bce0-03459112ffb9-os-release\") pod \"multus-additional-cni-plugins-8br7q\" (UID: \"51d01c0c-2588-432a-bce0-03459112ffb9\") " pod="openshift-multus/multus-additional-cni-plugins-8br7q" Jul 28 13:47:27.281267 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.281171 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/51d01c0c-2588-432a-bce0-03459112ffb9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8br7q\" (UID: \"51d01c0c-2588-432a-bce0-03459112ffb9\") " pod="openshift-multus/multus-additional-cni-plugins-8br7q" Jul 28 13:47:27.281267 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.281200 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0ebc30d5-9500-40ed-b7f6-0372a013a978-iptables-alerter-script\") pod \"iptables-alerter-j5v9s\" (UID: \"0ebc30d5-9500-40ed-b7f6-0372a013a978\") " pod="openshift-network-operator/iptables-alerter-j5v9s" Jul 28 13:47:27.281267 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.281225 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0ebc30d5-9500-40ed-b7f6-0372a013a978-host-slash\") pod \"iptables-alerter-j5v9s\" (UID: \"0ebc30d5-9500-40ed-b7f6-0372a013a978\") " pod="openshift-network-operator/iptables-alerter-j5v9s" Jul 28 13:47:27.281267 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.281250 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dnbr\" (UniqueName: \"kubernetes.io/projected/0ebc30d5-9500-40ed-b7f6-0372a013a978-kube-api-access-9dnbr\") pod \"iptables-alerter-j5v9s\" (UID: \"0ebc30d5-9500-40ed-b7f6-0372a013a978\") " pod="openshift-network-operator/iptables-alerter-j5v9s" Jul 28 13:47:27.281550 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.281428 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.282180 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.282157 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 28 13:47:27.282285 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.282195 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 28 13:47:27.282349 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.282335 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 28 13:47:27.282465 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.282450 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 28 13:47:27.282531 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.282462 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-gbm2r\"" Jul 28 13:47:27.282531 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.282450 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 28 13:47:27.282648 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.282464 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 28 13:47:27.282648 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.282450 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 28 13:47:27.282648 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.282596 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 28 13:47:27.282648 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.282606 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-4ktjr\"" Jul 28 13:47:27.282830 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.282701 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bgkv9" Jul 28 13:47:27.282938 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.282923 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 28 13:47:27.283700 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.283683 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 28 13:47:27.283808 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.283794 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 28 13:47:27.283877 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.283826 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jw2xt\"" Jul 28 13:47:27.284166 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.284148 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qvpwc" Jul 28 13:47:27.285582 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.285568 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 28 13:47:27.285678 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.285664 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.285772 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.285758 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 28 13:47:27.285976 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.285959 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4tvmq\"" Jul 28 13:47:27.287067 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.286864 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vvwjc" Jul 28 13:47:27.287067 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.286884 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 28 13:47:27.287067 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.286894 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 28 13:47:27.287067 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.286905 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 28 13:47:27.287067 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.286872 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-bdjvj\"" Jul 28 13:47:27.287874 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.287857 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 28 13:47:27.288050 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.288035 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-9kdpm\"" Jul 28 13:47:27.288123 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.288096 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:47:27.288175 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:27.288152 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dlx" podUID="f5efacaa-1da7-4ed7-8bfa-2e805507fde7" Jul 28 13:47:27.289418 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.289250 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-xj2j8\"" Jul 28 13:47:27.289418 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.289253 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 28 13:47:27.289603 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.289498 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 28 13:47:27.289603 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.289562 2550 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 28 13:47:27.289603 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.289570 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khglj" Jul 28 13:47:27.289753 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:27.289692 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khglj" podUID="7b621445-7c29-4e1f-a55e-bc7161202598" Jul 28 13:47:27.290914 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.290898 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z2sg" Jul 28 13:47:27.290974 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:27.290947 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z2sg" podUID="fbc15680-5aba-470f-b2ee-fcdf596983ae" Jul 28 13:47:27.299732 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.299708 2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 13:42:26 +0000 UTC" deadline="2027-12-23 03:36:50.63864969 +0000 UTC" Jul 28 13:47:27.299732 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.299730 2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12301h49m23.33892181s" Jul 28 13:47:27.313494 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.313477 2550 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-swljg" Jul 28 13:47:27.326033 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.326015 2550 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-swljg" Jul 28 13:47:27.375931 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.375904 2550 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 28 13:47:27.381840 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.381818 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpbcl\" (UniqueName: \"kubernetes.io/projected/7b621445-7c29-4e1f-a55e-bc7161202598-kube-api-access-lpbcl\") pod \"network-metrics-daemon-khglj\" (UID: \"7b621445-7c29-4e1f-a55e-bc7161202598\") " pod="openshift-multus/network-metrics-daemon-khglj" Jul 28 13:47:27.381967 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.381845 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-var-lib-openvswitch\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.381967 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.381864 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/266eea55-0f52-46d3-a006-8282aab58caa-etc-sysctl-d\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.381967 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.381910 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/266eea55-0f52-46d3-a006-8282aab58caa-etc-tuned\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.381967 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.381948 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/51d01c0c-2588-432a-bce0-03459112ffb9-os-release\") pod \"multus-additional-cni-plugins-8br7q\" (UID: \"51d01c0c-2588-432a-bce0-03459112ffb9\") " pod="openshift-multus/multus-additional-cni-plugins-8br7q" Jul 28 13:47:27.382159 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.381969 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/51d01c0c-2588-432a-bce0-03459112ffb9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8br7q\" (UID: \"51d01c0c-2588-432a-bce0-03459112ffb9\") " pod="openshift-multus/multus-additional-cni-plugins-8br7q" Jul 28 13:47:27.382159 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.381986 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-os-release\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.382159 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382000 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f5efacaa-1da7-4ed7-8bfa-2e805507fde7-kubelet-config\") pod \"global-pull-secret-syncer-89dlx\" (UID: \"f5efacaa-1da7-4ed7-8bfa-2e805507fde7\") " pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:47:27.382159 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382018 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a390d1e3-916f-43f8-b75f-940d222744a8-sys-fs\") pod \"aws-ebs-csi-driver-node-wktfs\" (UID: \"a390d1e3-916f-43f8-b75f-940d222744a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wktfs" Jul 28 13:47:27.382159 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382035 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzjvh\" (UniqueName: \"kubernetes.io/projected/fbc15680-5aba-470f-b2ee-fcdf596983ae-kube-api-access-fzjvh\") pod \"network-check-target-2z2sg\" (UID: \"fbc15680-5aba-470f-b2ee-fcdf596983ae\") " pod="openshift-network-diagnostics/network-check-target-2z2sg" Jul 28 13:47:27.382159 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382038 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/51d01c0c-2588-432a-bce0-03459112ffb9-os-release\") pod \"multus-additional-cni-plugins-8br7q\" (UID: \"51d01c0c-2588-432a-bce0-03459112ffb9\") " pod="openshift-multus/multus-additional-cni-plugins-8br7q" Jul 28 13:47:27.382159 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382071 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/266eea55-0f52-46d3-a006-8282aab58caa-etc-systemd\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.382159 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382139 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/266eea55-0f52-46d3-a006-8282aab58caa-sys\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.382566 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382168 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f498bb4f-ae84-4a84-ba7d-189d4a713911-ovnkube-config\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.382566 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382189 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f498bb4f-ae84-4a84-ba7d-189d4a713911-ovn-node-metrics-cert\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.382566 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382222 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/51d01c0c-2588-432a-bce0-03459112ffb9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8br7q\" (UID: \"51d01c0c-2588-432a-bce0-03459112ffb9\") " pod="openshift-multus/multus-additional-cni-plugins-8br7q" Jul 28 13:47:27.382566 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382269 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-system-cni-dir\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.382566 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382286 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-run-systemd\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.382566 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382299 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-node-log\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.382566 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382313 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/266eea55-0f52-46d3-a006-8282aab58caa-run\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.382566 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382328 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-log-socket\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.382566 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382344 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-host-run-ovn-kubernetes\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.382566 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382373 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2a365673-18ca-4ce3-9895-3e1f3e6bea6d-host\") pod \"node-ca-qvpwc\" (UID: \"2a365673-18ca-4ce3-9895-3e1f3e6bea6d\") " pod="openshift-image-registry/node-ca-qvpwc" Jul 28 13:47:27.382566 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382410 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/59ba325f-5a4f-442d-9c36-618faba03f71-hosts-file\") pod \"node-resolver-bgkv9\" (UID: \"59ba325f-5a4f-442d-9c36-618faba03f71\") " pod="openshift-dns/node-resolver-bgkv9" Jul 28 13:47:27.382566 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382430 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0ebc30d5-9500-40ed-b7f6-0372a013a978-host-slash\") pod \"iptables-alerter-j5v9s\" (UID: \"0ebc30d5-9500-40ed-b7f6-0372a013a978\") " pod="openshift-network-operator/iptables-alerter-j5v9s" Jul 28 13:47:27.382566 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382447 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dnbr\" (UniqueName: \"kubernetes.io/projected/0ebc30d5-9500-40ed-b7f6-0372a013a978-kube-api-access-9dnbr\") pod \"iptables-alerter-j5v9s\" (UID: \"0ebc30d5-9500-40ed-b7f6-0372a013a978\") " pod="openshift-network-operator/iptables-alerter-j5v9s" Jul 28 13:47:27.382566 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382462 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-host-var-lib-cni-bin\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.382566 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382482 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0ebc30d5-9500-40ed-b7f6-0372a013a978-host-slash\") pod \"iptables-alerter-j5v9s\" (UID: \"0ebc30d5-9500-40ed-b7f6-0372a013a978\") " pod="openshift-network-operator/iptables-alerter-j5v9s" Jul 28 13:47:27.382566 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382490 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-host-slash\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.382566 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382512 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-host-cni-bin\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.383358 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382532 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/55845350-bd96-44ac-ba5f-700cb9efdf60-konnectivity-ca\") pod \"konnectivity-agent-vvwjc\" (UID: \"55845350-bd96-44ac-ba5f-700cb9efdf60\") " pod="kube-system/konnectivity-agent-vvwjc" Jul 28 13:47:27.383358 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382565 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ln2r\" (UniqueName: \"kubernetes.io/projected/59ba325f-5a4f-442d-9c36-618faba03f71-kube-api-access-9ln2r\") pod \"node-resolver-bgkv9\" (UID: \"59ba325f-5a4f-442d-9c36-618faba03f71\") " pod="openshift-dns/node-resolver-bgkv9" Jul 28 13:47:27.383358 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382584 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/937ca4bc-863d-45d1-a83a-c497cba19b2c-multus-daemon-config\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.383358 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382601 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a390d1e3-916f-43f8-b75f-940d222744a8-socket-dir\") pod \"aws-ebs-csi-driver-node-wktfs\" (UID: \"a390d1e3-916f-43f8-b75f-940d222744a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wktfs" Jul 28 13:47:27.383358 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382616 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/266eea55-0f52-46d3-a006-8282aab58caa-etc-modprobe-d\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.383358 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382635 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/266eea55-0f52-46d3-a006-8282aab58caa-lib-modules\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.383358 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382671 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/937ca4bc-863d-45d1-a83a-c497cba19b2c-cni-binary-copy\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.383358 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382702 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0ebc30d5-9500-40ed-b7f6-0372a013a978-iptables-alerter-script\") pod \"iptables-alerter-j5v9s\" (UID: \"0ebc30d5-9500-40ed-b7f6-0372a013a978\") " pod="openshift-network-operator/iptables-alerter-j5v9s" Jul 28 13:47:27.383358 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382734 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/51d01c0c-2588-432a-bce0-03459112ffb9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8br7q\" (UID: \"51d01c0c-2588-432a-bce0-03459112ffb9\") " pod="openshift-multus/multus-additional-cni-plugins-8br7q" Jul 28 13:47:27.383358 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382755 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-host-run-netns\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.383358 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382770 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/266eea55-0f52-46d3-a006-8282aab58caa-var-lib-kubelet\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.383358 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382793 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-etc-kubernetes\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.383358 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382818 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5nhx\" (UniqueName: \"kubernetes.io/projected/937ca4bc-863d-45d1-a83a-c497cba19b2c-kube-api-access-h5nhx\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.383358 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382843 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a390d1e3-916f-43f8-b75f-940d222744a8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wktfs\" (UID: \"a390d1e3-916f-43f8-b75f-940d222744a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wktfs" Jul 28 13:47:27.383358 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382893 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxt55\" (UniqueName: \"kubernetes.io/projected/2a365673-18ca-4ce3-9895-3e1f3e6bea6d-kube-api-access-xxt55\") pod \"node-ca-qvpwc\" (UID: \"2a365673-18ca-4ce3-9895-3e1f3e6bea6d\") " pod="openshift-image-registry/node-ca-qvpwc" Jul 28 13:47:27.383358 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382920 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/51d01c0c-2588-432a-bce0-03459112ffb9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8br7q\" (UID: \"51d01c0c-2588-432a-bce0-03459112ffb9\") " pod="openshift-multus/multus-additional-cni-plugins-8br7q" Jul 28 13:47:27.384043 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382931 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.384043 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382963 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-multus-socket-dir-parent\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.384043 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.382987 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-host-var-lib-cni-multus\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.384043 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383005 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-etc-openvswitch\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.384043 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383025 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-multus-cni-dir\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.384043 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383044 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f5efacaa-1da7-4ed7-8bfa-2e805507fde7-dbus\") pod \"global-pull-secret-syncer-89dlx\" (UID: \"f5efacaa-1da7-4ed7-8bfa-2e805507fde7\") " pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:47:27.384043 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383060 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-run-openvswitch\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.384043 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383092 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a390d1e3-916f-43f8-b75f-940d222744a8-etc-selinux\") pod \"aws-ebs-csi-driver-node-wktfs\" (UID: \"a390d1e3-916f-43f8-b75f-940d222744a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wktfs" Jul 28 13:47:27.384043 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383111 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/266eea55-0f52-46d3-a006-8282aab58caa-etc-sysconfig\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.384043 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383125 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/266eea55-0f52-46d3-a006-8282aab58caa-etc-kubernetes\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.384043 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383140 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/266eea55-0f52-46d3-a006-8282aab58caa-etc-sysctl-conf\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.384043 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383163 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59ba325f-5a4f-442d-9c36-618faba03f71-tmp-dir\") pod \"node-resolver-bgkv9\" (UID: \"59ba325f-5a4f-442d-9c36-618faba03f71\") " pod="openshift-dns/node-resolver-bgkv9" Jul 28 13:47:27.384043 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383187 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f498bb4f-ae84-4a84-ba7d-189d4a713911-env-overrides\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.384043 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383213 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kz9j9\" (UniqueName: \"kubernetes.io/projected/51d01c0c-2588-432a-bce0-03459112ffb9-kube-api-access-kz9j9\") pod \"multus-additional-cni-plugins-8br7q\" (UID: \"51d01c0c-2588-432a-bce0-03459112ffb9\") " pod="openshift-multus/multus-additional-cni-plugins-8br7q" Jul 28 13:47:27.384043 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383229 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-host-kubelet\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.384043 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383244 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6plq6\" (UniqueName: \"kubernetes.io/projected/a390d1e3-916f-43f8-b75f-940d222744a8-kube-api-access-6plq6\") pod \"aws-ebs-csi-driver-node-wktfs\" (UID: \"a390d1e3-916f-43f8-b75f-940d222744a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wktfs" Jul 28 13:47:27.384634 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383266 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-run-ovn\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.384634 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383290 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f498bb4f-ae84-4a84-ba7d-189d4a713911-ovnkube-script-lib\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.384634 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383314 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/55845350-bd96-44ac-ba5f-700cb9efdf60-agent-certs\") pod \"konnectivity-agent-vvwjc\" (UID: \"55845350-bd96-44ac-ba5f-700cb9efdf60\") " pod="kube-system/konnectivity-agent-vvwjc" Jul 28 13:47:27.384634 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383336 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/266eea55-0f52-46d3-a006-8282aab58caa-host\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.384634 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383359 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/51d01c0c-2588-432a-bce0-03459112ffb9-system-cni-dir\") pod \"multus-additional-cni-plugins-8br7q\" (UID: \"51d01c0c-2588-432a-bce0-03459112ffb9\") " pod="openshift-multus/multus-additional-cni-plugins-8br7q" Jul 28 13:47:27.384634 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383379 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/51d01c0c-2588-432a-bce0-03459112ffb9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8br7q\" (UID: \"51d01c0c-2588-432a-bce0-03459112ffb9\") " pod="openshift-multus/multus-additional-cni-plugins-8br7q" Jul 28 13:47:27.384634 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383380 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-host-run-k8s-cni-cncf-io\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.384634 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383433 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-host-var-lib-kubelet\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.384634 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383453 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/51d01c0c-2588-432a-bce0-03459112ffb9-system-cni-dir\") pod \"multus-additional-cni-plugins-8br7q\" (UID: \"51d01c0c-2588-432a-bce0-03459112ffb9\") " pod="openshift-multus/multus-additional-cni-plugins-8br7q" Jul 28 13:47:27.384634 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383459 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f5efacaa-1da7-4ed7-8bfa-2e805507fde7-original-pull-secret\") pod \"global-pull-secret-syncer-89dlx\" (UID: \"f5efacaa-1da7-4ed7-8bfa-2e805507fde7\") " pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:47:27.384634 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383454 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/51d01c0c-2588-432a-bce0-03459112ffb9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8br7q\" (UID: \"51d01c0c-2588-432a-bce0-03459112ffb9\") " pod="openshift-multus/multus-additional-cni-plugins-8br7q" Jul 28 13:47:27.384634 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383502 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a390d1e3-916f-43f8-b75f-940d222744a8-registration-dir\") pod \"aws-ebs-csi-driver-node-wktfs\" (UID: \"a390d1e3-916f-43f8-b75f-940d222744a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wktfs" Jul 28 13:47:27.384634 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383526 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a390d1e3-916f-43f8-b75f-940d222744a8-device-dir\") pod \"aws-ebs-csi-driver-node-wktfs\" (UID: \"a390d1e3-916f-43f8-b75f-940d222744a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wktfs" Jul 28 13:47:27.384634 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383579 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/51d01c0c-2588-432a-bce0-03459112ffb9-cnibin\") pod \"multus-additional-cni-plugins-8br7q\" (UID: \"51d01c0c-2588-432a-bce0-03459112ffb9\") " pod="openshift-multus/multus-additional-cni-plugins-8br7q" Jul 28 13:47:27.384634 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383598 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b621445-7c29-4e1f-a55e-bc7161202598-metrics-certs\") pod \"network-metrics-daemon-khglj\" (UID: \"7b621445-7c29-4e1f-a55e-bc7161202598\") " pod="openshift-multus/network-metrics-daemon-khglj" Jul 28 13:47:27.384634 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383629 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-systemd-units\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.385084 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383662 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-host-cni-netd\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.385084 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383664 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/51d01c0c-2588-432a-bce0-03459112ffb9-cnibin\") pod \"multus-additional-cni-plugins-8br7q\" (UID: \"51d01c0c-2588-432a-bce0-03459112ffb9\") " pod="openshift-multus/multus-additional-cni-plugins-8br7q" Jul 28 13:47:27.385084 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383693 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwj7l\" (UniqueName: \"kubernetes.io/projected/266eea55-0f52-46d3-a006-8282aab58caa-kube-api-access-cwj7l\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.385084 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383713 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-cnibin\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.385084 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383727 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-hostroot\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.385084 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383746 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whdhx\" (UniqueName: \"kubernetes.io/projected/f498bb4f-ae84-4a84-ba7d-189d4a713911-kube-api-access-whdhx\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.385084 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383776 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2a365673-18ca-4ce3-9895-3e1f3e6bea6d-serviceca\") pod \"node-ca-qvpwc\" (UID: \"2a365673-18ca-4ce3-9895-3e1f3e6bea6d\") " pod="openshift-image-registry/node-ca-qvpwc" Jul 28 13:47:27.385084 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383823 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/266eea55-0f52-46d3-a006-8282aab58caa-tmp\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.385084 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383855 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/51d01c0c-2588-432a-bce0-03459112ffb9-cni-binary-copy\") pod \"multus-additional-cni-plugins-8br7q\" (UID: \"51d01c0c-2588-432a-bce0-03459112ffb9\") " pod="openshift-multus/multus-additional-cni-plugins-8br7q" Jul 28 13:47:27.385084 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383886 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-host-run-netns\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.385084 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383909 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-multus-conf-dir\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.385084 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.383935 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-host-run-multus-certs\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.385084 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.384252 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/51d01c0c-2588-432a-bce0-03459112ffb9-cni-binary-copy\") pod \"multus-additional-cni-plugins-8br7q\" (UID: \"51d01c0c-2588-432a-bce0-03459112ffb9\") " pod="openshift-multus/multus-additional-cni-plugins-8br7q" Jul 28 13:47:27.385084 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.384547 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0ebc30d5-9500-40ed-b7f6-0372a013a978-iptables-alerter-script\") pod \"iptables-alerter-j5v9s\" (UID: \"0ebc30d5-9500-40ed-b7f6-0372a013a978\") " pod="openshift-network-operator/iptables-alerter-j5v9s" Jul 28 13:47:27.394285 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.394258 2550 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 28 13:47:27.397130 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.397110 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dnbr\" (UniqueName: \"kubernetes.io/projected/0ebc30d5-9500-40ed-b7f6-0372a013a978-kube-api-access-9dnbr\") pod \"iptables-alerter-j5v9s\" (UID: \"0ebc30d5-9500-40ed-b7f6-0372a013a978\") " pod="openshift-network-operator/iptables-alerter-j5v9s" Jul 28 13:47:27.397212 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.397114 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kz9j9\" (UniqueName: \"kubernetes.io/projected/51d01c0c-2588-432a-bce0-03459112ffb9-kube-api-access-kz9j9\") pod \"multus-additional-cni-plugins-8br7q\" (UID: \"51d01c0c-2588-432a-bce0-03459112ffb9\") " pod="openshift-multus/multus-additional-cni-plugins-8br7q" Jul 28 13:47:27.484213 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484179 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-multus-cni-dir\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.484213 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484211 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f5efacaa-1da7-4ed7-8bfa-2e805507fde7-dbus\") pod \"global-pull-secret-syncer-89dlx\" (UID: \"f5efacaa-1da7-4ed7-8bfa-2e805507fde7\") " pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:47:27.484454 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484230 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-run-openvswitch\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.484454 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484253 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a390d1e3-916f-43f8-b75f-940d222744a8-etc-selinux\") pod \"aws-ebs-csi-driver-node-wktfs\" (UID: \"a390d1e3-916f-43f8-b75f-940d222744a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wktfs" Jul 28 13:47:27.484454 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484273 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/266eea55-0f52-46d3-a006-8282aab58caa-etc-sysconfig\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.484454 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484300 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/266eea55-0f52-46d3-a006-8282aab58caa-etc-kubernetes\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.484454 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484307 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-multus-cni-dir\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.484454 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484317 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f5efacaa-1da7-4ed7-8bfa-2e805507fde7-dbus\") pod \"global-pull-secret-syncer-89dlx\" (UID: \"f5efacaa-1da7-4ed7-8bfa-2e805507fde7\") " pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:47:27.484454 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484323 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/266eea55-0f52-46d3-a006-8282aab58caa-etc-sysctl-conf\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.484454 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484319 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-run-openvswitch\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.484454 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484347 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59ba325f-5a4f-442d-9c36-618faba03f71-tmp-dir\") pod \"node-resolver-bgkv9\" (UID: \"59ba325f-5a4f-442d-9c36-618faba03f71\") " pod="openshift-dns/node-resolver-bgkv9" Jul 28 13:47:27.484454 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484370 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f498bb4f-ae84-4a84-ba7d-189d4a713911-env-overrides\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.484454 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484367 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/266eea55-0f52-46d3-a006-8282aab58caa-etc-sysconfig\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.484454 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484369 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/266eea55-0f52-46d3-a006-8282aab58caa-etc-kubernetes\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.484454 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484409 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-host-kubelet\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.484454 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484432 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6plq6\" (UniqueName: \"kubernetes.io/projected/a390d1e3-916f-43f8-b75f-940d222744a8-kube-api-access-6plq6\") pod \"aws-ebs-csi-driver-node-wktfs\" (UID: \"a390d1e3-916f-43f8-b75f-940d222744a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wktfs" Jul 28 13:47:27.484454 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484431 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a390d1e3-916f-43f8-b75f-940d222744a8-etc-selinux\") pod \"aws-ebs-csi-driver-node-wktfs\" (UID: \"a390d1e3-916f-43f8-b75f-940d222744a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wktfs" Jul 28 13:47:27.484454 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484452 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-run-ovn\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.485213 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484459 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-host-kubelet\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.485213 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484488 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/266eea55-0f52-46d3-a006-8282aab58caa-etc-sysctl-conf\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.485213 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484492 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f498bb4f-ae84-4a84-ba7d-189d4a713911-ovnkube-script-lib\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.485213 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484520 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/55845350-bd96-44ac-ba5f-700cb9efdf60-agent-certs\") pod \"konnectivity-agent-vvwjc\" (UID: \"55845350-bd96-44ac-ba5f-700cb9efdf60\") " pod="kube-system/konnectivity-agent-vvwjc" Jul 28 13:47:27.485213 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484536 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/266eea55-0f52-46d3-a006-8282aab58caa-host\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.485213 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484558 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-host-run-k8s-cni-cncf-io\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.485213 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484496 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-run-ovn\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.485213 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484582 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-host-var-lib-kubelet\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.485213 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484620 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-host-run-k8s-cni-cncf-io\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.485213 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484639 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f5efacaa-1da7-4ed7-8bfa-2e805507fde7-original-pull-secret\") pod \"global-pull-secret-syncer-89dlx\" (UID: \"f5efacaa-1da7-4ed7-8bfa-2e805507fde7\") " pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:47:27.485213 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484664 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/266eea55-0f52-46d3-a006-8282aab58caa-host\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.485213 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484667 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a390d1e3-916f-43f8-b75f-940d222744a8-registration-dir\") pod \"aws-ebs-csi-driver-node-wktfs\" (UID: \"a390d1e3-916f-43f8-b75f-940d222744a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wktfs" Jul 28 13:47:27.485213 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484692 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a390d1e3-916f-43f8-b75f-940d222744a8-device-dir\") pod \"aws-ebs-csi-driver-node-wktfs\" (UID: \"a390d1e3-916f-43f8-b75f-940d222744a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wktfs" Jul 28 13:47:27.485213 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484699 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-host-var-lib-kubelet\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.485213 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484715 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b621445-7c29-4e1f-a55e-bc7161202598-metrics-certs\") pod \"network-metrics-daemon-khglj\" (UID: \"7b621445-7c29-4e1f-a55e-bc7161202598\") " pod="openshift-multus/network-metrics-daemon-khglj" Jul 28 13:47:27.485213 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484742 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-systemd-units\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.485213 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484742 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a390d1e3-916f-43f8-b75f-940d222744a8-registration-dir\") pod \"aws-ebs-csi-driver-node-wktfs\" (UID: \"a390d1e3-916f-43f8-b75f-940d222744a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wktfs" Jul 28 13:47:27.486068 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484770 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-host-cni-netd\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.486068 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484789 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a390d1e3-916f-43f8-b75f-940d222744a8-device-dir\") pod \"aws-ebs-csi-driver-node-wktfs\" (UID: \"a390d1e3-916f-43f8-b75f-940d222744a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wktfs" Jul 28 13:47:27.486068 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484793 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59ba325f-5a4f-442d-9c36-618faba03f71-tmp-dir\") pod \"node-resolver-bgkv9\" (UID: \"59ba325f-5a4f-442d-9c36-618faba03f71\") " pod="openshift-dns/node-resolver-bgkv9" Jul 28 13:47:27.486068 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484799 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwj7l\" (UniqueName: \"kubernetes.io/projected/266eea55-0f52-46d3-a006-8282aab58caa-kube-api-access-cwj7l\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.486068 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:27.484794 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 13:47:27.486068 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484821 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-systemd-units\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.486068 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:27.484857 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 13:47:27.486068 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:27.484886 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5efacaa-1da7-4ed7-8bfa-2e805507fde7-original-pull-secret podName:f5efacaa-1da7-4ed7-8bfa-2e805507fde7 nodeName:}" failed. No retries permitted until 2026-07-28 13:47:27.98486387 +0000 UTC m=+2.120060766 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f5efacaa-1da7-4ed7-8bfa-2e805507fde7-original-pull-secret") pod "global-pull-secret-syncer-89dlx" (UID: "f5efacaa-1da7-4ed7-8bfa-2e805507fde7") : object "kube-system"/"original-pull-secret" not registered Jul 28 13:47:27.486068 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484856 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-host-cni-netd\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.486068 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:27.484914 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b621445-7c29-4e1f-a55e-bc7161202598-metrics-certs podName:7b621445-7c29-4e1f-a55e-bc7161202598 nodeName:}" failed. No retries permitted until 2026-07-28 13:47:27.984904447 +0000 UTC m=+2.120101320 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b621445-7c29-4e1f-a55e-bc7161202598-metrics-certs") pod "network-metrics-daemon-khglj" (UID: "7b621445-7c29-4e1f-a55e-bc7161202598") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 13:47:27.486068 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484935 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-cnibin\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.486068 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484964 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-hostroot\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.486068 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.484990 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whdhx\" (UniqueName: \"kubernetes.io/projected/f498bb4f-ae84-4a84-ba7d-189d4a713911-kube-api-access-whdhx\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.486068 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485016 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2a365673-18ca-4ce3-9895-3e1f3e6bea6d-serviceca\") pod \"node-ca-qvpwc\" (UID: \"2a365673-18ca-4ce3-9895-3e1f3e6bea6d\") " pod="openshift-image-registry/node-ca-qvpwc" Jul 28 13:47:27.486068 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485020 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-cnibin\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.486068 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485039 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/266eea55-0f52-46d3-a006-8282aab58caa-tmp\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.486068 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485022 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-hostroot\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.486994 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485065 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-host-run-netns\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.486994 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485082 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f498bb4f-ae84-4a84-ba7d-189d4a713911-ovnkube-script-lib\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.486994 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485089 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-multus-conf-dir\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.486994 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485137 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-multus-conf-dir\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.486994 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485202 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-host-run-netns\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.486994 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485235 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-host-run-multus-certs\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.486994 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485264 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpbcl\" (UniqueName: \"kubernetes.io/projected/7b621445-7c29-4e1f-a55e-bc7161202598-kube-api-access-lpbcl\") pod \"network-metrics-daemon-khglj\" (UID: \"7b621445-7c29-4e1f-a55e-bc7161202598\") " pod="openshift-multus/network-metrics-daemon-khglj" Jul 28 13:47:27.486994 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485290 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-var-lib-openvswitch\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.486994 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485301 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-host-run-multus-certs\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.486994 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485312 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/266eea55-0f52-46d3-a006-8282aab58caa-etc-sysctl-d\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.486994 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485350 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/266eea55-0f52-46d3-a006-8282aab58caa-etc-tuned\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.486994 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485378 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-os-release\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.486994 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485418 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f5efacaa-1da7-4ed7-8bfa-2e805507fde7-kubelet-config\") pod \"global-pull-secret-syncer-89dlx\" (UID: \"f5efacaa-1da7-4ed7-8bfa-2e805507fde7\") " pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:47:27.486994 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485443 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a390d1e3-916f-43f8-b75f-940d222744a8-sys-fs\") pod \"aws-ebs-csi-driver-node-wktfs\" (UID: \"a390d1e3-916f-43f8-b75f-940d222744a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wktfs" Jul 28 13:47:27.486994 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485472 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzjvh\" (UniqueName: \"kubernetes.io/projected/fbc15680-5aba-470f-b2ee-fcdf596983ae-kube-api-access-fzjvh\") pod \"network-check-target-2z2sg\" (UID: \"fbc15680-5aba-470f-b2ee-fcdf596983ae\") " pod="openshift-network-diagnostics/network-check-target-2z2sg" Jul 28 13:47:27.486994 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485497 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/266eea55-0f52-46d3-a006-8282aab58caa-etc-systemd\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.486994 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485522 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/266eea55-0f52-46d3-a006-8282aab58caa-sys\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.486994 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485532 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-var-lib-openvswitch\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.487767 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485523 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2a365673-18ca-4ce3-9895-3e1f3e6bea6d-serviceca\") pod \"node-ca-qvpwc\" (UID: \"2a365673-18ca-4ce3-9895-3e1f3e6bea6d\") " pod="openshift-image-registry/node-ca-qvpwc" Jul 28 13:47:27.487767 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485551 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f498bb4f-ae84-4a84-ba7d-189d4a713911-ovnkube-config\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.487767 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485556 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f498bb4f-ae84-4a84-ba7d-189d4a713911-env-overrides\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.487767 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485576 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f498bb4f-ae84-4a84-ba7d-189d4a713911-ovn-node-metrics-cert\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.487767 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485587 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/266eea55-0f52-46d3-a006-8282aab58caa-etc-sysctl-d\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.487767 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485615 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-system-cni-dir\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.487767 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485639 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a390d1e3-916f-43f8-b75f-940d222744a8-sys-fs\") pod \"aws-ebs-csi-driver-node-wktfs\" (UID: \"a390d1e3-916f-43f8-b75f-940d222744a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wktfs" Jul 28 13:47:27.487767 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485684 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-system-cni-dir\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.487767 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485641 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-run-systemd\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.487767 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485722 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-node-log\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.487767 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485747 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/266eea55-0f52-46d3-a006-8282aab58caa-run\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.487767 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485769 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/266eea55-0f52-46d3-a006-8282aab58caa-sys\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.487767 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485695 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/266eea55-0f52-46d3-a006-8282aab58caa-etc-systemd\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.487767 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485809 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-log-socket\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.487767 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485815 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-node-log\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.487767 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485772 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-log-socket\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.487767 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485725 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-run-systemd\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.487767 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485849 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-host-run-ovn-kubernetes\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.488260 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485868 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/266eea55-0f52-46d3-a006-8282aab58caa-run\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.488260 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485877 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2a365673-18ca-4ce3-9895-3e1f3e6bea6d-host\") pod \"node-ca-qvpwc\" (UID: \"2a365673-18ca-4ce3-9895-3e1f3e6bea6d\") " pod="openshift-image-registry/node-ca-qvpwc" Jul 28 13:47:27.488260 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485905 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/59ba325f-5a4f-442d-9c36-618faba03f71-hosts-file\") pod \"node-resolver-bgkv9\" (UID: \"59ba325f-5a4f-442d-9c36-618faba03f71\") " pod="openshift-dns/node-resolver-bgkv9" Jul 28 13:47:27.488260 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485921 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-os-release\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.488260 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485949 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-host-run-ovn-kubernetes\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.488260 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485964 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2a365673-18ca-4ce3-9895-3e1f3e6bea6d-host\") pod \"node-ca-qvpwc\" (UID: \"2a365673-18ca-4ce3-9895-3e1f3e6bea6d\") " pod="openshift-image-registry/node-ca-qvpwc" Jul 28 13:47:27.488260 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485978 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/59ba325f-5a4f-442d-9c36-618faba03f71-hosts-file\") pod \"node-resolver-bgkv9\" (UID: \"59ba325f-5a4f-442d-9c36-618faba03f71\") " pod="openshift-dns/node-resolver-bgkv9" Jul 28 13:47:27.488260 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.485977 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-host-var-lib-cni-bin\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.488260 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.486006 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-host-var-lib-cni-bin\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.488260 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.486010 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-host-slash\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.488260 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.486023 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f5efacaa-1da7-4ed7-8bfa-2e805507fde7-kubelet-config\") pod \"global-pull-secret-syncer-89dlx\" (UID: \"f5efacaa-1da7-4ed7-8bfa-2e805507fde7\") " pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:47:27.488260 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.486039 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-host-cni-bin\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.488260 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.486064 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-host-slash\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.488260 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.486064 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/55845350-bd96-44ac-ba5f-700cb9efdf60-konnectivity-ca\") pod \"konnectivity-agent-vvwjc\" (UID: \"55845350-bd96-44ac-ba5f-700cb9efdf60\") " pod="kube-system/konnectivity-agent-vvwjc" Jul 28 13:47:27.488260 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.486119 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-host-cni-bin\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.488260 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.486220 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ln2r\" (UniqueName: \"kubernetes.io/projected/59ba325f-5a4f-442d-9c36-618faba03f71-kube-api-access-9ln2r\") pod \"node-resolver-bgkv9\" (UID: \"59ba325f-5a4f-442d-9c36-618faba03f71\") " pod="openshift-dns/node-resolver-bgkv9" Jul 28 13:47:27.488260 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.486251 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/937ca4bc-863d-45d1-a83a-c497cba19b2c-multus-daemon-config\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.488260 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.486276 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a390d1e3-916f-43f8-b75f-940d222744a8-socket-dir\") pod \"aws-ebs-csi-driver-node-wktfs\" (UID: \"a390d1e3-916f-43f8-b75f-940d222744a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wktfs" Jul 28 13:47:27.488904 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.486302 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/266eea55-0f52-46d3-a006-8282aab58caa-etc-modprobe-d\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.488904 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.486326 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/266eea55-0f52-46d3-a006-8282aab58caa-lib-modules\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.488904 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.486348 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/937ca4bc-863d-45d1-a83a-c497cba19b2c-cni-binary-copy\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.488904 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.486375 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-host-run-netns\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.488904 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.486407 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f498bb4f-ae84-4a84-ba7d-189d4a713911-ovnkube-config\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.488904 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.486416 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/266eea55-0f52-46d3-a006-8282aab58caa-var-lib-kubelet\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.488904 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.486442 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-etc-kubernetes\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.488904 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.486467 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5nhx\" (UniqueName: \"kubernetes.io/projected/937ca4bc-863d-45d1-a83a-c497cba19b2c-kube-api-access-h5nhx\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.488904 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.486483 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a390d1e3-916f-43f8-b75f-940d222744a8-socket-dir\") pod \"aws-ebs-csi-driver-node-wktfs\" (UID: \"a390d1e3-916f-43f8-b75f-940d222744a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wktfs" Jul 28 13:47:27.488904 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.486494 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a390d1e3-916f-43f8-b75f-940d222744a8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wktfs\" (UID: \"a390d1e3-916f-43f8-b75f-940d222744a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wktfs" Jul 28 13:47:27.488904 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.486533 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxt55\" (UniqueName: \"kubernetes.io/projected/2a365673-18ca-4ce3-9895-3e1f3e6bea6d-kube-api-access-xxt55\") pod \"node-ca-qvpwc\" (UID: \"2a365673-18ca-4ce3-9895-3e1f3e6bea6d\") " pod="openshift-image-registry/node-ca-qvpwc" Jul 28 13:47:27.488904 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.486557 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-host-run-netns\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.488904 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.486567 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/55845350-bd96-44ac-ba5f-700cb9efdf60-konnectivity-ca\") pod \"konnectivity-agent-vvwjc\" (UID: \"55845350-bd96-44ac-ba5f-700cb9efdf60\") " pod="kube-system/konnectivity-agent-vvwjc" Jul 28 13:47:27.488904 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.486627 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/266eea55-0f52-46d3-a006-8282aab58caa-var-lib-kubelet\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.488904 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.486571 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.488904 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.486666 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-multus-socket-dir-parent\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.488904 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.486718 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-multus-socket-dir-parent\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.489377 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.486698 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-host-var-lib-cni-multus\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.489377 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.486757 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a390d1e3-916f-43f8-b75f-940d222744a8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wktfs\" (UID: \"a390d1e3-916f-43f8-b75f-940d222744a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wktfs" Jul 28 13:47:27.489377 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.486767 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-etc-openvswitch\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.489377 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.486793 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.489377 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.486837 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f498bb4f-ae84-4a84-ba7d-189d4a713911-etc-openvswitch\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.489377 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.486851 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/266eea55-0f52-46d3-a006-8282aab58caa-etc-modprobe-d\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.489377 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.486853 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-host-var-lib-cni-multus\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.489377 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.486897 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/937ca4bc-863d-45d1-a83a-c497cba19b2c-etc-kubernetes\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.489377 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.486950 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/266eea55-0f52-46d3-a006-8282aab58caa-lib-modules\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.489377 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.487042 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/937ca4bc-863d-45d1-a83a-c497cba19b2c-multus-daemon-config\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.489377 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.487367 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/937ca4bc-863d-45d1-a83a-c497cba19b2c-cni-binary-copy\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.489377 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.487530 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/266eea55-0f52-46d3-a006-8282aab58caa-tmp\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.489377 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.487560 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/55845350-bd96-44ac-ba5f-700cb9efdf60-agent-certs\") pod \"konnectivity-agent-vvwjc\" (UID: \"55845350-bd96-44ac-ba5f-700cb9efdf60\") " pod="kube-system/konnectivity-agent-vvwjc" Jul 28 13:47:27.489377 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.488473 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f498bb4f-ae84-4a84-ba7d-189d4a713911-ovn-node-metrics-cert\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.489377 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.488677 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/266eea55-0f52-46d3-a006-8282aab58caa-etc-tuned\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.493469 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:27.493450 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 13:47:27.493469 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:27.493471 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 13:47:27.493593 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:27.493484 2550 projected.go:196] Error preparing data for projected volume kube-api-access-fzjvh for pod openshift-network-diagnostics/network-check-target-2z2sg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 13:47:27.493593 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:27.493542 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fbc15680-5aba-470f-b2ee-fcdf596983ae-kube-api-access-fzjvh podName:fbc15680-5aba-470f-b2ee-fcdf596983ae nodeName:}" failed. No retries permitted until 2026-07-28 13:47:27.993527613 +0000 UTC m=+2.128724499 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-fzjvh" (UniqueName: "kubernetes.io/projected/fbc15680-5aba-470f-b2ee-fcdf596983ae-kube-api-access-fzjvh") pod "network-check-target-2z2sg" (UID: "fbc15680-5aba-470f-b2ee-fcdf596983ae") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 13:47:27.494124 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.494109 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwj7l\" (UniqueName: \"kubernetes.io/projected/266eea55-0f52-46d3-a006-8282aab58caa-kube-api-access-cwj7l\") pod \"tuned-v2wxp\" (UID: \"266eea55-0f52-46d3-a006-8282aab58caa\") " pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.494752 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.494732 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whdhx\" (UniqueName: \"kubernetes.io/projected/f498bb4f-ae84-4a84-ba7d-189d4a713911-kube-api-access-whdhx\") pod \"ovnkube-node-4nb4j\" (UID: \"f498bb4f-ae84-4a84-ba7d-189d4a713911\") " pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.495008 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.494969 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6plq6\" (UniqueName: \"kubernetes.io/projected/a390d1e3-916f-43f8-b75f-940d222744a8-kube-api-access-6plq6\") pod \"aws-ebs-csi-driver-node-wktfs\" (UID: \"a390d1e3-916f-43f8-b75f-940d222744a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wktfs" Jul 28 13:47:27.496627 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.496597 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxt55\" (UniqueName: \"kubernetes.io/projected/2a365673-18ca-4ce3-9895-3e1f3e6bea6d-kube-api-access-xxt55\") pod \"node-ca-qvpwc\" (UID: \"2a365673-18ca-4ce3-9895-3e1f3e6bea6d\") " pod="openshift-image-registry/node-ca-qvpwc" Jul 28 13:47:27.496772 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.496755 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ln2r\" (UniqueName: \"kubernetes.io/projected/59ba325f-5a4f-442d-9c36-618faba03f71-kube-api-access-9ln2r\") pod \"node-resolver-bgkv9\" (UID: \"59ba325f-5a4f-442d-9c36-618faba03f71\") " pod="openshift-dns/node-resolver-bgkv9" Jul 28 13:47:27.497135 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.497119 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5nhx\" (UniqueName: \"kubernetes.io/projected/937ca4bc-863d-45d1-a83a-c497cba19b2c-kube-api-access-h5nhx\") pod \"multus-6d5r6\" (UID: \"937ca4bc-863d-45d1-a83a-c497cba19b2c\") " pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.497626 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.497611 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpbcl\" (UniqueName: \"kubernetes.io/projected/7b621445-7c29-4e1f-a55e-bc7161202598-kube-api-access-lpbcl\") pod \"network-metrics-daemon-khglj\" (UID: \"7b621445-7c29-4e1f-a55e-bc7161202598\") " pod="openshift-multus/network-metrics-daemon-khglj" Jul 28 13:47:27.608959 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.608857 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8br7q" Jul 28 13:47:27.613012 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.612995 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-j5v9s" Jul 28 13:47:27.641943 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.641913 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:27.655929 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.655905 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wktfs" Jul 28 13:47:27.671031 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.671008 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" Jul 28 13:47:27.683745 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.683722 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bgkv9" Jul 28 13:47:27.704637 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.704604 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qvpwc" Jul 28 13:47:27.719587 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.719564 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6d5r6" Jul 28 13:47:27.724404 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.724371 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vvwjc" Jul 28 13:47:27.790205 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.790171 2550 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 13:47:27.790566 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.790543 2550 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 13:47:27.990753 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.990680 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f5efacaa-1da7-4ed7-8bfa-2e805507fde7-original-pull-secret\") pod \"global-pull-secret-syncer-89dlx\" (UID: \"f5efacaa-1da7-4ed7-8bfa-2e805507fde7\") " pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:47:27.990753 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:27.990713 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b621445-7c29-4e1f-a55e-bc7161202598-metrics-certs\") pod \"network-metrics-daemon-khglj\" (UID: \"7b621445-7c29-4e1f-a55e-bc7161202598\") " pod="openshift-multus/network-metrics-daemon-khglj" Jul 28 13:47:27.990932 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:27.990819 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 13:47:27.990932 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:27.990877 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5efacaa-1da7-4ed7-8bfa-2e805507fde7-original-pull-secret podName:f5efacaa-1da7-4ed7-8bfa-2e805507fde7 nodeName:}" failed. No retries permitted until 2026-07-28 13:47:28.990862366 +0000 UTC m=+3.126059241 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f5efacaa-1da7-4ed7-8bfa-2e805507fde7-original-pull-secret") pod "global-pull-secret-syncer-89dlx" (UID: "f5efacaa-1da7-4ed7-8bfa-2e805507fde7") : object "kube-system"/"original-pull-secret" not registered Jul 28 13:47:27.990932 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:27.990819 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 13:47:27.991043 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:27.990939 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b621445-7c29-4e1f-a55e-bc7161202598-metrics-certs podName:7b621445-7c29-4e1f-a55e-bc7161202598 nodeName:}" failed. No retries permitted until 2026-07-28 13:47:28.990924087 +0000 UTC m=+3.126120968 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b621445-7c29-4e1f-a55e-bc7161202598-metrics-certs") pod "network-metrics-daemon-khglj" (UID: "7b621445-7c29-4e1f-a55e-bc7161202598") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 13:47:28.091302 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:28.091267 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzjvh\" (UniqueName: \"kubernetes.io/projected/fbc15680-5aba-470f-b2ee-fcdf596983ae-kube-api-access-fzjvh\") pod \"network-check-target-2z2sg\" (UID: \"fbc15680-5aba-470f-b2ee-fcdf596983ae\") " pod="openshift-network-diagnostics/network-check-target-2z2sg" Jul 28 13:47:28.091531 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:28.091384 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 13:47:28.091531 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:28.091416 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 13:47:28.091531 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:28.091425 2550 projected.go:196] Error preparing data for projected volume kube-api-access-fzjvh for pod openshift-network-diagnostics/network-check-target-2z2sg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 13:47:28.091531 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:28.091470 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fbc15680-5aba-470f-b2ee-fcdf596983ae-kube-api-access-fzjvh podName:fbc15680-5aba-470f-b2ee-fcdf596983ae nodeName:}" failed. No retries permitted until 2026-07-28 13:47:29.091455862 +0000 UTC m=+3.226652735 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-fzjvh" (UniqueName: "kubernetes.io/projected/fbc15680-5aba-470f-b2ee-fcdf596983ae-kube-api-access-fzjvh") pod "network-check-target-2z2sg" (UID: "fbc15680-5aba-470f-b2ee-fcdf596983ae") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 13:47:28.185656 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:28.185607 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod021d5cd1abd768b313395dd23c6c4054.slice/crio-c7b4f15d284e425025d023de8e9497445b3634b589a40dbe4ce493b7537ef992 WatchSource:0}: Error finding container c7b4f15d284e425025d023de8e9497445b3634b589a40dbe4ce493b7537ef992: Status 404 returned error can't find the container with id c7b4f15d284e425025d023de8e9497445b3634b589a40dbe4ce493b7537ef992 Jul 28 13:47:28.186170 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:28.186152 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf498bb4f_ae84_4a84_ba7d_189d4a713911.slice/crio-760d72a724c285f08e1cbea7583791a76cad2a7443e23b1f3d155ff43d04ba1c WatchSource:0}: Error finding container 760d72a724c285f08e1cbea7583791a76cad2a7443e23b1f3d155ff43d04ba1c: Status 404 returned error can't find the container with id 760d72a724c285f08e1cbea7583791a76cad2a7443e23b1f3d155ff43d04ba1c Jul 28 13:47:28.189734 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:28.189708 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda390d1e3_916f_43f8_b75f_940d222744a8.slice/crio-6d72b8eb22064cc994e58da3d534f64cdc192cef394e05e8145385ecbb8284d9 WatchSource:0}: Error finding container 6d72b8eb22064cc994e58da3d534f64cdc192cef394e05e8145385ecbb8284d9: Status 404 returned error can't find the container with id 6d72b8eb22064cc994e58da3d534f64cdc192cef394e05e8145385ecbb8284d9 Jul 28 13:47:28.196084 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:28.196067 2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 13:47:28.204118 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:28.204096 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a365673_18ca_4ce3_9895_3e1f3e6bea6d.slice/crio-ddcc43d7af366c293bcf4d79741dacd82df14b70ba35f641304a355fad47aac1 WatchSource:0}: Error finding container ddcc43d7af366c293bcf4d79741dacd82df14b70ba35f641304a355fad47aac1: Status 404 returned error can't find the container with id ddcc43d7af366c293bcf4d79741dacd82df14b70ba35f641304a355fad47aac1 Jul 28 13:47:28.211356 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:28.211330 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51d01c0c_2588_432a_bce0_03459112ffb9.slice/crio-81abc35d2e72bf11ffff74f6f13d6bfe4f72d39c2455c1bc6157cbac8af08a2e WatchSource:0}: Error finding container 81abc35d2e72bf11ffff74f6f13d6bfe4f72d39c2455c1bc6157cbac8af08a2e: Status 404 returned error can't find the container with id 81abc35d2e72bf11ffff74f6f13d6bfe4f72d39c2455c1bc6157cbac8af08a2e Jul 28 13:47:28.242503 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:28.242448 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd99d1c9d97e7e4c15b53cfee9d40b63c.slice/crio-3876d9634b2d185a3da512e4281672b0c2151ef46c6a5e98eb50b023065bd442 WatchSource:0}: Error finding container 3876d9634b2d185a3da512e4281672b0c2151ef46c6a5e98eb50b023065bd442: Status 404 returned error can't find the container with id 3876d9634b2d185a3da512e4281672b0c2151ef46c6a5e98eb50b023065bd442 Jul 28 13:47:28.328533 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:28.328470 2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 13:42:27 +0000 UTC" deadline="2028-01-08 22:01:53.356556585 +0000 UTC" Jul 28 13:47:28.328533 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:28.328521 2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12704h14m25.028038203s" Jul 28 13:47:28.364527 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:28.364496 2550 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 13:47:28.458987 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:28.458953 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55845350_bd96_44ac_ba5f_700cb9efdf60.slice/crio-0b45da00c5af228ddc9ea7f51d69d69e2f159225269cf9a8e39f76009103cf32 WatchSource:0}: Error finding container 0b45da00c5af228ddc9ea7f51d69d69e2f159225269cf9a8e39f76009103cf32: Status 404 returned error can't find the container with id 0b45da00c5af228ddc9ea7f51d69d69e2f159225269cf9a8e39f76009103cf32 Jul 28 13:47:28.472041 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:28.472020 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z2sg" Jul 28 13:47:28.472137 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:28.472117 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z2sg" podUID="fbc15680-5aba-470f-b2ee-fcdf596983ae" Jul 28 13:47:28.477804 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:28.477760 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-63.ec2.internal" event={"ID":"d99d1c9d97e7e4c15b53cfee9d40b63c","Type":"ContainerStarted","Data":"3876d9634b2d185a3da512e4281672b0c2151ef46c6a5e98eb50b023065bd442"} Jul 28 13:47:28.478771 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:28.478749 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8br7q" event={"ID":"51d01c0c-2588-432a-bce0-03459112ffb9","Type":"ContainerStarted","Data":"81abc35d2e72bf11ffff74f6f13d6bfe4f72d39c2455c1bc6157cbac8af08a2e"} Jul 28 13:47:28.479753 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:28.479734 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qvpwc" event={"ID":"2a365673-18ca-4ce3-9895-3e1f3e6bea6d","Type":"ContainerStarted","Data":"ddcc43d7af366c293bcf4d79741dacd82df14b70ba35f641304a355fad47aac1"} Jul 28 13:47:28.480728 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:28.480713 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wktfs" event={"ID":"a390d1e3-916f-43f8-b75f-940d222744a8","Type":"ContainerStarted","Data":"6d72b8eb22064cc994e58da3d534f64cdc192cef394e05e8145385ecbb8284d9"} Jul 28 13:47:28.481701 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:28.481682 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" event={"ID":"f498bb4f-ae84-4a84-ba7d-189d4a713911","Type":"ContainerStarted","Data":"760d72a724c285f08e1cbea7583791a76cad2a7443e23b1f3d155ff43d04ba1c"} Jul 28 13:47:28.482659 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:28.482641 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-63.ec2.internal" event={"ID":"021d5cd1abd768b313395dd23c6c4054","Type":"ContainerStarted","Data":"c7b4f15d284e425025d023de8e9497445b3634b589a40dbe4ce493b7537ef992"} Jul 28 13:47:28.483608 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:28.483592 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vvwjc" event={"ID":"55845350-bd96-44ac-ba5f-700cb9efdf60","Type":"ContainerStarted","Data":"0b45da00c5af228ddc9ea7f51d69d69e2f159225269cf9a8e39f76009103cf32"} Jul 28 13:47:28.886176 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:28.886109 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod937ca4bc_863d_45d1_a83a_c497cba19b2c.slice/crio-57a6f481b15454140491c2490607a4edfb9121aa9157c41814d7887b27053629 WatchSource:0}: Error finding container 57a6f481b15454140491c2490607a4edfb9121aa9157c41814d7887b27053629: Status 404 returned error can't find the container with id 57a6f481b15454140491c2490607a4edfb9121aa9157c41814d7887b27053629 Jul 28 13:47:28.894410 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:28.894363 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod266eea55_0f52_46d3_a006_8282aab58caa.slice/crio-2e63ed9e9a32b14d967a25491a5bff0584deaad8b6b77fa8d48a56fcad73ee8d WatchSource:0}: Error finding container 2e63ed9e9a32b14d967a25491a5bff0584deaad8b6b77fa8d48a56fcad73ee8d: Status 404 returned error can't find the container with id 2e63ed9e9a32b14d967a25491a5bff0584deaad8b6b77fa8d48a56fcad73ee8d Jul 28 13:47:28.999460 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:28.998646 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f5efacaa-1da7-4ed7-8bfa-2e805507fde7-original-pull-secret\") pod \"global-pull-secret-syncer-89dlx\" (UID: \"f5efacaa-1da7-4ed7-8bfa-2e805507fde7\") " pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:47:28.999460 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:28.998700 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b621445-7c29-4e1f-a55e-bc7161202598-metrics-certs\") pod \"network-metrics-daemon-khglj\" (UID: \"7b621445-7c29-4e1f-a55e-bc7161202598\") " pod="openshift-multus/network-metrics-daemon-khglj" Jul 28 13:47:28.999460 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:28.998889 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 13:47:28.999460 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:28.998950 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b621445-7c29-4e1f-a55e-bc7161202598-metrics-certs podName:7b621445-7c29-4e1f-a55e-bc7161202598 nodeName:}" failed. No retries permitted until 2026-07-28 13:47:30.998929903 +0000 UTC m=+5.134126774 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b621445-7c29-4e1f-a55e-bc7161202598-metrics-certs") pod "network-metrics-daemon-khglj" (UID: "7b621445-7c29-4e1f-a55e-bc7161202598") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 13:47:28.999460 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:28.999357 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 13:47:28.999460 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:28.999423 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5efacaa-1da7-4ed7-8bfa-2e805507fde7-original-pull-secret podName:f5efacaa-1da7-4ed7-8bfa-2e805507fde7 nodeName:}" failed. No retries permitted until 2026-07-28 13:47:30.999407802 +0000 UTC m=+5.134604679 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f5efacaa-1da7-4ed7-8bfa-2e805507fde7-original-pull-secret") pod "global-pull-secret-syncer-89dlx" (UID: "f5efacaa-1da7-4ed7-8bfa-2e805507fde7") : object "kube-system"/"original-pull-secret" not registered Jul 28 13:47:29.099661 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:29.099621 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzjvh\" (UniqueName: \"kubernetes.io/projected/fbc15680-5aba-470f-b2ee-fcdf596983ae-kube-api-access-fzjvh\") pod \"network-check-target-2z2sg\" (UID: \"fbc15680-5aba-470f-b2ee-fcdf596983ae\") " pod="openshift-network-diagnostics/network-check-target-2z2sg" Jul 28 13:47:29.099849 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:29.099809 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 13:47:29.099849 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:29.099831 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 13:47:29.099849 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:29.099845 2550 projected.go:196] Error preparing data for projected volume kube-api-access-fzjvh for pod openshift-network-diagnostics/network-check-target-2z2sg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 13:47:29.099999 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:29.099903 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fbc15680-5aba-470f-b2ee-fcdf596983ae-kube-api-access-fzjvh podName:fbc15680-5aba-470f-b2ee-fcdf596983ae nodeName:}" failed. No retries permitted until 2026-07-28 13:47:31.099885242 +0000 UTC m=+5.235082115 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-fzjvh" (UniqueName: "kubernetes.io/projected/fbc15680-5aba-470f-b2ee-fcdf596983ae-kube-api-access-fzjvh") pod "network-check-target-2z2sg" (UID: "fbc15680-5aba-470f-b2ee-fcdf596983ae") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 13:47:29.328951 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:29.328862 2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 13:42:27 +0000 UTC" deadline="2028-01-06 02:48:48.296266182 +0000 UTC" Jul 28 13:47:29.328951 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:29.328907 2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12637h1m18.967364846s" Jul 28 13:47:29.341664 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:29.341630 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ebc30d5_9500_40ed_b7f6_0372a013a978.slice/crio-0d0ce5050f0af8ab7dd058e70297973834c8a7518318ba140460454389dc7700 WatchSource:0}: Error finding container 0d0ce5050f0af8ab7dd058e70297973834c8a7518318ba140460454389dc7700: Status 404 returned error can't find the container with id 0d0ce5050f0af8ab7dd058e70297973834c8a7518318ba140460454389dc7700 Jul 28 13:47:29.375484 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:29.375454 2550 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 13:47:29.473302 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:29.472617 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khglj" Jul 28 13:47:29.473302 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:29.472760 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khglj" podUID="7b621445-7c29-4e1f-a55e-bc7161202598" Jul 28 13:47:29.473302 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:29.473151 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:47:29.473302 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:29.473250 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dlx" podUID="f5efacaa-1da7-4ed7-8bfa-2e805507fde7" Jul 28 13:47:29.505714 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:29.505675 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" event={"ID":"266eea55-0f52-46d3-a006-8282aab58caa","Type":"ContainerStarted","Data":"2e63ed9e9a32b14d967a25491a5bff0584deaad8b6b77fa8d48a56fcad73ee8d"} Jul 28 13:47:29.514808 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:29.514776 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6d5r6" event={"ID":"937ca4bc-863d-45d1-a83a-c497cba19b2c","Type":"ContainerStarted","Data":"57a6f481b15454140491c2490607a4edfb9121aa9157c41814d7887b27053629"} Jul 28 13:47:29.518407 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:29.518361 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-j5v9s" event={"ID":"0ebc30d5-9500-40ed-b7f6-0372a013a978","Type":"ContainerStarted","Data":"0d0ce5050f0af8ab7dd058e70297973834c8a7518318ba140460454389dc7700"} Jul 28 13:47:29.708664 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:47:29.708419 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59ba325f_5a4f_442d_9c36_618faba03f71.slice/crio-febad44618e9bebd0795e19299515d77b270f473d351674e3e3f8c3040fa4b30 WatchSource:0}: Error finding container febad44618e9bebd0795e19299515d77b270f473d351674e3e3f8c3040fa4b30: Status 404 returned error can't find the container with id febad44618e9bebd0795e19299515d77b270f473d351674e3e3f8c3040fa4b30 Jul 28 13:47:30.472438 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:30.472405 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z2sg" Jul 28 13:47:30.472877 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:30.472584 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z2sg" podUID="fbc15680-5aba-470f-b2ee-fcdf596983ae" Jul 28 13:47:30.523461 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:30.523369 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bgkv9" event={"ID":"59ba325f-5a4f-442d-9c36-618faba03f71","Type":"ContainerStarted","Data":"febad44618e9bebd0795e19299515d77b270f473d351674e3e3f8c3040fa4b30"} Jul 28 13:47:31.016316 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:31.016271 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b621445-7c29-4e1f-a55e-bc7161202598-metrics-certs\") pod \"network-metrics-daemon-khglj\" (UID: \"7b621445-7c29-4e1f-a55e-bc7161202598\") " pod="openshift-multus/network-metrics-daemon-khglj" Jul 28 13:47:31.016532 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:31.016412 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f5efacaa-1da7-4ed7-8bfa-2e805507fde7-original-pull-secret\") pod \"global-pull-secret-syncer-89dlx\" (UID: \"f5efacaa-1da7-4ed7-8bfa-2e805507fde7\") " pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:47:31.016621 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:31.016533 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 13:47:31.016621 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:31.016596 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5efacaa-1da7-4ed7-8bfa-2e805507fde7-original-pull-secret podName:f5efacaa-1da7-4ed7-8bfa-2e805507fde7 nodeName:}" failed. No retries permitted until 2026-07-28 13:47:35.016576837 +0000 UTC m=+9.151773710 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f5efacaa-1da7-4ed7-8bfa-2e805507fde7-original-pull-secret") pod "global-pull-secret-syncer-89dlx" (UID: "f5efacaa-1da7-4ed7-8bfa-2e805507fde7") : object "kube-system"/"original-pull-secret" not registered Jul 28 13:47:31.017104 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:31.017006 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 13:47:31.017104 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:31.017064 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b621445-7c29-4e1f-a55e-bc7161202598-metrics-certs podName:7b621445-7c29-4e1f-a55e-bc7161202598 nodeName:}" failed. No retries permitted until 2026-07-28 13:47:35.017047445 +0000 UTC m=+9.152244317 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b621445-7c29-4e1f-a55e-bc7161202598-metrics-certs") pod "network-metrics-daemon-khglj" (UID: "7b621445-7c29-4e1f-a55e-bc7161202598") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 13:47:31.117700 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:31.117662 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzjvh\" (UniqueName: \"kubernetes.io/projected/fbc15680-5aba-470f-b2ee-fcdf596983ae-kube-api-access-fzjvh\") pod \"network-check-target-2z2sg\" (UID: \"fbc15680-5aba-470f-b2ee-fcdf596983ae\") " pod="openshift-network-diagnostics/network-check-target-2z2sg" Jul 28 13:47:31.117892 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:31.117850 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 13:47:31.117892 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:31.117875 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 13:47:31.117892 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:31.117887 2550 projected.go:196] Error preparing data for projected volume kube-api-access-fzjvh for pod openshift-network-diagnostics/network-check-target-2z2sg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 13:47:31.118041 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:31.117957 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fbc15680-5aba-470f-b2ee-fcdf596983ae-kube-api-access-fzjvh podName:fbc15680-5aba-470f-b2ee-fcdf596983ae nodeName:}" failed. No retries permitted until 2026-07-28 13:47:35.117937168 +0000 UTC m=+9.253134057 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-fzjvh" (UniqueName: "kubernetes.io/projected/fbc15680-5aba-470f-b2ee-fcdf596983ae-kube-api-access-fzjvh") pod "network-check-target-2z2sg" (UID: "fbc15680-5aba-470f-b2ee-fcdf596983ae") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 13:47:31.472329 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:31.472223 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khglj" Jul 28 13:47:31.472329 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:31.472267 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:47:31.472858 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:31.472378 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khglj" podUID="7b621445-7c29-4e1f-a55e-bc7161202598" Jul 28 13:47:31.472858 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:31.472508 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dlx" podUID="f5efacaa-1da7-4ed7-8bfa-2e805507fde7" Jul 28 13:47:32.472702 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:32.472187 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z2sg" Jul 28 13:47:32.472702 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:32.472320 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z2sg" podUID="fbc15680-5aba-470f-b2ee-fcdf596983ae" Jul 28 13:47:33.471950 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:33.471915 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khglj" Jul 28 13:47:33.472147 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:33.472067 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khglj" podUID="7b621445-7c29-4e1f-a55e-bc7161202598" Jul 28 13:47:33.472620 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:33.472471 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:47:33.472620 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:33.472571 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dlx" podUID="f5efacaa-1da7-4ed7-8bfa-2e805507fde7" Jul 28 13:47:34.472587 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:34.472455 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z2sg" Jul 28 13:47:34.472785 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:34.472628 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z2sg" podUID="fbc15680-5aba-470f-b2ee-fcdf596983ae" Jul 28 13:47:35.054319 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:35.054172 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f5efacaa-1da7-4ed7-8bfa-2e805507fde7-original-pull-secret\") pod \"global-pull-secret-syncer-89dlx\" (UID: \"f5efacaa-1da7-4ed7-8bfa-2e805507fde7\") " pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:47:35.054319 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:35.054223 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b621445-7c29-4e1f-a55e-bc7161202598-metrics-certs\") pod \"network-metrics-daemon-khglj\" (UID: \"7b621445-7c29-4e1f-a55e-bc7161202598\") " pod="openshift-multus/network-metrics-daemon-khglj" Jul 28 13:47:35.054319 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:35.054312 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 13:47:35.054633 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:35.054374 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5efacaa-1da7-4ed7-8bfa-2e805507fde7-original-pull-secret podName:f5efacaa-1da7-4ed7-8bfa-2e805507fde7 nodeName:}" failed. No retries permitted until 2026-07-28 13:47:43.054356697 +0000 UTC m=+17.189553584 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f5efacaa-1da7-4ed7-8bfa-2e805507fde7-original-pull-secret") pod "global-pull-secret-syncer-89dlx" (UID: "f5efacaa-1da7-4ed7-8bfa-2e805507fde7") : object "kube-system"/"original-pull-secret" not registered Jul 28 13:47:35.054633 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:35.054423 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 13:47:35.054633 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:35.054511 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b621445-7c29-4e1f-a55e-bc7161202598-metrics-certs podName:7b621445-7c29-4e1f-a55e-bc7161202598 nodeName:}" failed. No retries permitted until 2026-07-28 13:47:43.054492461 +0000 UTC m=+17.189689334 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b621445-7c29-4e1f-a55e-bc7161202598-metrics-certs") pod "network-metrics-daemon-khglj" (UID: "7b621445-7c29-4e1f-a55e-bc7161202598") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 13:47:35.155344 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:35.155301 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzjvh\" (UniqueName: \"kubernetes.io/projected/fbc15680-5aba-470f-b2ee-fcdf596983ae-kube-api-access-fzjvh\") pod \"network-check-target-2z2sg\" (UID: \"fbc15680-5aba-470f-b2ee-fcdf596983ae\") " pod="openshift-network-diagnostics/network-check-target-2z2sg" Jul 28 13:47:35.155546 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:35.155529 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 13:47:35.155614 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:35.155558 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 13:47:35.155614 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:35.155574 2550 projected.go:196] Error preparing data for projected volume kube-api-access-fzjvh for pod openshift-network-diagnostics/network-check-target-2z2sg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 13:47:35.155719 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:35.155641 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fbc15680-5aba-470f-b2ee-fcdf596983ae-kube-api-access-fzjvh podName:fbc15680-5aba-470f-b2ee-fcdf596983ae nodeName:}" failed. No retries permitted until 2026-07-28 13:47:43.155620774 +0000 UTC m=+17.290817667 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-fzjvh" (UniqueName: "kubernetes.io/projected/fbc15680-5aba-470f-b2ee-fcdf596983ae-kube-api-access-fzjvh") pod "network-check-target-2z2sg" (UID: "fbc15680-5aba-470f-b2ee-fcdf596983ae") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 13:47:35.472143 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:35.471618 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khglj" Jul 28 13:47:35.472143 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:35.471662 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:47:35.472143 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:35.471758 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khglj" podUID="7b621445-7c29-4e1f-a55e-bc7161202598" Jul 28 13:47:35.472143 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:35.471850 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dlx" podUID="f5efacaa-1da7-4ed7-8bfa-2e805507fde7" Jul 28 13:47:36.473831 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:36.473361 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z2sg" Jul 28 13:47:36.473831 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:36.473488 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z2sg" podUID="fbc15680-5aba-470f-b2ee-fcdf596983ae" Jul 28 13:47:37.471950 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:37.471913 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khglj" Jul 28 13:47:37.472144 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:37.471933 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:47:37.472144 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:37.472069 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khglj" podUID="7b621445-7c29-4e1f-a55e-bc7161202598" Jul 28 13:47:37.472251 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:37.472164 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dlx" podUID="f5efacaa-1da7-4ed7-8bfa-2e805507fde7" Jul 28 13:47:38.473067 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:38.473029 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z2sg" Jul 28 13:47:38.473523 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:38.473225 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z2sg" podUID="fbc15680-5aba-470f-b2ee-fcdf596983ae" Jul 28 13:47:39.472085 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:39.472049 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khglj" Jul 28 13:47:39.472276 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:39.472092 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:47:39.472276 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:39.472195 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khglj" podUID="7b621445-7c29-4e1f-a55e-bc7161202598" Jul 28 13:47:39.472407 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:39.472311 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dlx" podUID="f5efacaa-1da7-4ed7-8bfa-2e805507fde7" Jul 28 13:47:40.472014 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:40.471986 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z2sg" Jul 28 13:47:40.472497 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:40.472103 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z2sg" podUID="fbc15680-5aba-470f-b2ee-fcdf596983ae" Jul 28 13:47:41.472516 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:41.472478 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khglj" Jul 28 13:47:41.472516 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:41.472511 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:47:41.473019 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:41.472622 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khglj" podUID="7b621445-7c29-4e1f-a55e-bc7161202598" Jul 28 13:47:41.473019 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:41.472751 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dlx" podUID="f5efacaa-1da7-4ed7-8bfa-2e805507fde7" Jul 28 13:47:42.472585 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:42.472548 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z2sg" Jul 28 13:47:42.473055 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:42.472690 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z2sg" podUID="fbc15680-5aba-470f-b2ee-fcdf596983ae" Jul 28 13:47:43.117809 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:43.115940 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f5efacaa-1da7-4ed7-8bfa-2e805507fde7-original-pull-secret\") pod \"global-pull-secret-syncer-89dlx\" (UID: \"f5efacaa-1da7-4ed7-8bfa-2e805507fde7\") " pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:47:43.117809 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:43.116011 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b621445-7c29-4e1f-a55e-bc7161202598-metrics-certs\") pod \"network-metrics-daemon-khglj\" (UID: \"7b621445-7c29-4e1f-a55e-bc7161202598\") " pod="openshift-multus/network-metrics-daemon-khglj" Jul 28 13:47:43.117809 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:43.116215 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 13:47:43.117809 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:43.116303 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b621445-7c29-4e1f-a55e-bc7161202598-metrics-certs podName:7b621445-7c29-4e1f-a55e-bc7161202598 nodeName:}" failed. No retries permitted until 2026-07-28 13:47:59.116283204 +0000 UTC m=+33.251480079 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b621445-7c29-4e1f-a55e-bc7161202598-metrics-certs") pod "network-metrics-daemon-khglj" (UID: "7b621445-7c29-4e1f-a55e-bc7161202598") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 13:47:43.117809 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:43.116404 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 13:47:43.117809 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:43.116460 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5efacaa-1da7-4ed7-8bfa-2e805507fde7-original-pull-secret podName:f5efacaa-1da7-4ed7-8bfa-2e805507fde7 nodeName:}" failed. No retries permitted until 2026-07-28 13:47:59.116447258 +0000 UTC m=+33.251644139 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f5efacaa-1da7-4ed7-8bfa-2e805507fde7-original-pull-secret") pod "global-pull-secret-syncer-89dlx" (UID: "f5efacaa-1da7-4ed7-8bfa-2e805507fde7") : object "kube-system"/"original-pull-secret" not registered Jul 28 13:47:43.217347 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:43.217316 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzjvh\" (UniqueName: \"kubernetes.io/projected/fbc15680-5aba-470f-b2ee-fcdf596983ae-kube-api-access-fzjvh\") pod \"network-check-target-2z2sg\" (UID: \"fbc15680-5aba-470f-b2ee-fcdf596983ae\") " pod="openshift-network-diagnostics/network-check-target-2z2sg" Jul 28 13:47:43.217527 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:43.217490 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 13:47:43.217527 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:43.217509 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 13:47:43.217527 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:43.217519 2550 projected.go:196] Error preparing data for projected volume kube-api-access-fzjvh for pod openshift-network-diagnostics/network-check-target-2z2sg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 13:47:43.217649 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:43.217569 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fbc15680-5aba-470f-b2ee-fcdf596983ae-kube-api-access-fzjvh podName:fbc15680-5aba-470f-b2ee-fcdf596983ae nodeName:}" failed. No retries permitted until 2026-07-28 13:47:59.217555673 +0000 UTC m=+33.352752543 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-fzjvh" (UniqueName: "kubernetes.io/projected/fbc15680-5aba-470f-b2ee-fcdf596983ae-kube-api-access-fzjvh") pod "network-check-target-2z2sg" (UID: "fbc15680-5aba-470f-b2ee-fcdf596983ae") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 13:47:43.472647 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:43.472569 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khglj" Jul 28 13:47:43.473045 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:43.472701 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khglj" podUID="7b621445-7c29-4e1f-a55e-bc7161202598" Jul 28 13:47:43.473045 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:43.472750 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:47:43.473045 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:43.472842 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dlx" podUID="f5efacaa-1da7-4ed7-8bfa-2e805507fde7" Jul 28 13:47:44.471963 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:44.471934 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z2sg" Jul 28 13:47:44.472163 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:44.472063 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z2sg" podUID="fbc15680-5aba-470f-b2ee-fcdf596983ae" Jul 28 13:47:45.472273 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:45.472233 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:47:45.472684 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:45.472279 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khglj" Jul 28 13:47:45.472684 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:45.472373 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dlx" podUID="f5efacaa-1da7-4ed7-8bfa-2e805507fde7" Jul 28 13:47:45.472684 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:45.472502 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khglj" podUID="7b621445-7c29-4e1f-a55e-bc7161202598" Jul 28 13:47:46.473101 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:46.473060 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z2sg" Jul 28 13:47:46.473532 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:46.473152 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z2sg" podUID="fbc15680-5aba-470f-b2ee-fcdf596983ae" Jul 28 13:47:47.471860 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:47.471825 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:47:47.472018 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:47.471825 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khglj" Jul 28 13:47:47.472018 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:47.471955 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dlx" podUID="f5efacaa-1da7-4ed7-8bfa-2e805507fde7" Jul 28 13:47:47.472115 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:47.472015 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khglj" podUID="7b621445-7c29-4e1f-a55e-bc7161202598" Jul 28 13:47:48.472702 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:48.472374 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z2sg" Jul 28 13:47:48.473432 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:48.472774 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z2sg" podUID="fbc15680-5aba-470f-b2ee-fcdf596983ae" Jul 28 13:47:48.561214 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:48.561182 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6d5r6" event={"ID":"937ca4bc-863d-45d1-a83a-c497cba19b2c","Type":"ContainerStarted","Data":"b7d2403bb16d3f4c9cb8222f1fd9a5b214be517e0704672d659983a7fc3ba892"} Jul 28 13:47:48.562567 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:48.562526 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bgkv9" event={"ID":"59ba325f-5a4f-442d-9c36-618faba03f71","Type":"ContainerStarted","Data":"9ef911c6ef1d5ee1edb2ddc48c28b484fc36a07f42df262a82b2d5bab6871b94"} Jul 28 13:47:48.563856 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:48.563832 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vvwjc" event={"ID":"55845350-bd96-44ac-ba5f-700cb9efdf60","Type":"ContainerStarted","Data":"a3532215e38ac9415081d692e881f967ed68e0c3f970e0b320f2dd445721ec85"} Jul 28 13:47:48.565286 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:48.565261 2550 generic.go:356] "Generic (PLEG): container finished" podID="51d01c0c-2588-432a-bce0-03459112ffb9" containerID="345ee0d378a04bbf0e76f8f1ec3f469ac9c5574c3ad0167f8b0676738c885845" exitCode=0 Jul 28 13:47:48.565490 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:48.565341 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8br7q" event={"ID":"51d01c0c-2588-432a-bce0-03459112ffb9","Type":"ContainerDied","Data":"345ee0d378a04bbf0e76f8f1ec3f469ac9c5574c3ad0167f8b0676738c885845"} Jul 28 13:47:48.566884 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:48.566863 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qvpwc" event={"ID":"2a365673-18ca-4ce3-9895-3e1f3e6bea6d","Type":"ContainerStarted","Data":"eea69cd18065264429a0f28682d064ed987b9804e5d807f9a2c7e92f42dc26e9"} Jul 28 13:47:48.568285 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:48.568262 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wktfs" event={"ID":"a390d1e3-916f-43f8-b75f-940d222744a8","Type":"ContainerStarted","Data":"22ab9eee548b6b679a558c3429c92b176953dd0c2c7b66aa6e2e0bd091ecb71b"} Jul 28 13:47:48.569613 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:48.569596 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-63.ec2.internal" event={"ID":"d99d1c9d97e7e4c15b53cfee9d40b63c","Type":"ContainerStarted","Data":"5b3946a03cd1aeea346faec2a65566fd470a8bcbe64f2c957ad361ec2318dea2"} Jul 28 13:47:48.572566 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:48.572548 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4nb4j_f498bb4f-ae84-4a84-ba7d-189d4a713911/ovn-acl-logging/0.log" Jul 28 13:47:48.572887 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:48.572868 2550 generic.go:356] "Generic (PLEG): container finished" podID="f498bb4f-ae84-4a84-ba7d-189d4a713911" containerID="4218457c1b4e91612607ab1b7c11ef8a386c71a16c37ec23512d9f247541a015" exitCode=1 Jul 28 13:47:48.572944 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:48.572930 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" event={"ID":"f498bb4f-ae84-4a84-ba7d-189d4a713911","Type":"ContainerStarted","Data":"937684cc2a6c3cdd1612fc99f57a9356d8f944c8cd48549785d7ba0e1a6ae0c8"} Jul 28 13:47:48.572985 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:48.572978 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" event={"ID":"f498bb4f-ae84-4a84-ba7d-189d4a713911","Type":"ContainerStarted","Data":"98026376448cf89a5ac70656555a3682d4d327fd6cb1609d94e35a948d3335fa"} Jul 28 13:47:48.573030 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:48.573019 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" event={"ID":"f498bb4f-ae84-4a84-ba7d-189d4a713911","Type":"ContainerStarted","Data":"5b7c4fcb6bd8271299691d096016d5b754d839242437b11dc670c49512fd1740"} Jul 28 13:47:48.573075 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:48.573062 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" event={"ID":"f498bb4f-ae84-4a84-ba7d-189d4a713911","Type":"ContainerStarted","Data":"848a4a276ea5af656f7c71cb2a8e2f33bab70b8c191c862172c05c5d8f241e86"} Jul 28 13:47:48.573117 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:48.573092 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" event={"ID":"f498bb4f-ae84-4a84-ba7d-189d4a713911","Type":"ContainerDied","Data":"4218457c1b4e91612607ab1b7c11ef8a386c71a16c37ec23512d9f247541a015"} Jul 28 13:47:48.573173 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:48.573126 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" event={"ID":"f498bb4f-ae84-4a84-ba7d-189d4a713911","Type":"ContainerStarted","Data":"d83a67a221f424ef6f4016a68616f5af986cc4b156cf1755c995e15182da19e3"} Jul 28 13:47:48.574264 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:48.574246 2550 generic.go:356] "Generic (PLEG): container finished" podID="021d5cd1abd768b313395dd23c6c4054" containerID="a55a35fa7d077fa567a5b5a7da4b3a09edc6b015084789f02dd0a39ae0310a28" exitCode=0 Jul 28 13:47:48.574324 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:48.574306 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-63.ec2.internal" event={"ID":"021d5cd1abd768b313395dd23c6c4054","Type":"ContainerDied","Data":"a55a35fa7d077fa567a5b5a7da4b3a09edc6b015084789f02dd0a39ae0310a28"} Jul 28 13:47:48.575572 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:48.575551 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" event={"ID":"266eea55-0f52-46d3-a006-8282aab58caa","Type":"ContainerStarted","Data":"de71f450df212cebee1da654509068452241e62540c8951aa1a21766cd1e6adc"} Jul 28 13:47:48.582483 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:48.582435 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-6d5r6" podStartSLOduration=3.517046382 podStartE2EDuration="22.582420338s" totalImagesPullingTime="19.065373956s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 13:47:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:47:48.580972752 +0000 UTC m=+22.716169643" watchObservedRunningTime="2026-07-28 13:47:48.582420338 +0000 UTC m=+22.717617231" Jul 28 13:47:48.596410 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:48.596348 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-63.ec2.internal" podStartSLOduration=21.596338597 podStartE2EDuration="21.596338597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 13:47:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:47:48.596062624 +0000 UTC m=+22.731259519" watchObservedRunningTime="2026-07-28 13:47:48.596338597 +0000 UTC m=+22.731535488" Jul 28 13:47:48.612798 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:48.612757 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vvwjc" podStartSLOduration=3.633166835 podStartE2EDuration="22.612744822s" totalImagesPullingTime="18.979577987s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:47:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:47:48.612129796 +0000 UTC m=+22.747326688" watchObservedRunningTime="2026-07-28 13:47:48.612744822 +0000 UTC m=+22.747941713" Jul 28 13:47:48.674306 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:48.674195 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-v2wxp" podStartSLOduration=4.027370089 podStartE2EDuration="22.674179217s" totalImagesPullingTime="18.646809128s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 13:47:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:47:48.657495824 +0000 UTC m=+22.792692715" watchObservedRunningTime="2026-07-28 13:47:48.674179217 +0000 UTC m=+22.809376109" Jul 28 13:47:48.674306 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:48.674284 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-bgkv9" podStartSLOduration=4.950686177 podStartE2EDuration="22.674280338s" totalImagesPullingTime="17.723594161s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 13:47:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:47:48.674066582 +0000 UTC m=+22.809263473" watchObservedRunningTime="2026-07-28 13:47:48.674280338 +0000 UTC m=+22.809477230" Jul 28 13:47:48.691737 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:48.691691 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qvpwc" podStartSLOduration=3.4570343279999998 podStartE2EDuration="22.69167615s" totalImagesPullingTime="19.234641822s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 13:47:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:47:48.691235758 +0000 UTC m=+22.826432655" watchObservedRunningTime="2026-07-28 13:47:48.69167615 +0000 UTC m=+22.826873042" Jul 28 13:47:49.471665 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:49.471635 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khglj" Jul 28 13:47:49.471845 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:49.471773 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khglj" podUID="7b621445-7c29-4e1f-a55e-bc7161202598" Jul 28 13:47:49.471845 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:49.471832 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:47:49.472000 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:49.471956 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dlx" podUID="f5efacaa-1da7-4ed7-8bfa-2e805507fde7" Jul 28 13:47:49.579586 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:49.579537 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-j5v9s" event={"ID":"0ebc30d5-9500-40ed-b7f6-0372a013a978","Type":"ContainerStarted","Data":"4d39da9d99e340667e7af39bfe63ce66ca1019e487308e441e5c02b73f49ad8a"} Jul 28 13:47:49.598274 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:49.598237 2550 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 28 13:47:49.601059 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:49.601019 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-j5v9s" podStartSLOduration=5.402237568 podStartE2EDuration="23.601004007s" totalImagesPullingTime="18.198766439s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 13:47:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:47:49.600150726 +0000 UTC m=+23.735347622" watchObservedRunningTime="2026-07-28 13:47:49.601004007 +0000 UTC m=+23.736200896" Jul 28 13:47:50.356068 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:50.355967 2550 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T13:47:49.598271243Z","UUID":"28b6533b-9486-4931-ba54-89fb31b35d28","Handler":null,"Name":"","Endpoint":""} Jul 28 13:47:50.357589 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:50.357548 2550 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 28 13:47:50.357589 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:50.357591 2550 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 28 13:47:50.472433 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:50.472385 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z2sg" Jul 28 13:47:50.472611 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:50.472524 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z2sg" podUID="fbc15680-5aba-470f-b2ee-fcdf596983ae" Jul 28 13:47:50.584093 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:50.584061 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-63.ec2.internal" event={"ID":"021d5cd1abd768b313395dd23c6c4054","Type":"ContainerStarted","Data":"96a814a27fb7b6e1f44e59369e2eaf36f47e801104bc85da4812a37bab2b16e1"} Jul 28 13:47:50.586235 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:50.586208 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wktfs" event={"ID":"a390d1e3-916f-43f8-b75f-940d222744a8","Type":"ContainerStarted","Data":"75e6fb95818f50d466df83bd00897de4617b47481fd6d2113cabf00c3106f5a1"} Jul 28 13:47:50.605129 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:50.605087 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-63.ec2.internal" podStartSLOduration=23.605073907 podStartE2EDuration="23.605073907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 13:47:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:47:50.605036429 +0000 UTC m=+24.740233322" watchObservedRunningTime="2026-07-28 13:47:50.605073907 +0000 UTC m=+24.740270800" Jul 28 13:47:51.472147 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:51.472110 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khglj" Jul 28 13:47:51.472332 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:51.472246 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khglj" podUID="7b621445-7c29-4e1f-a55e-bc7161202598" Jul 28 13:47:51.472567 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:51.472541 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:47:51.472708 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:51.472659 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dlx" podUID="f5efacaa-1da7-4ed7-8bfa-2e805507fde7" Jul 28 13:47:51.594743 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:51.594714 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4nb4j_f498bb4f-ae84-4a84-ba7d-189d4a713911/ovn-acl-logging/0.log" Jul 28 13:47:51.595161 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:51.595107 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" event={"ID":"f498bb4f-ae84-4a84-ba7d-189d4a713911","Type":"ContainerStarted","Data":"702460eae0e9e67101fdaaa797ba5ad6623c1fdac7de12ed0ce370039b251525"} Jul 28 13:47:51.988280 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:51.988251 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vvwjc" Jul 28 13:47:52.058069 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:52.058036 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vvwjc" Jul 28 13:47:52.058670 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:52.058649 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vvwjc" Jul 28 13:47:52.472012 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:52.471933 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z2sg" Jul 28 13:47:52.472170 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:52.472060 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z2sg" podUID="fbc15680-5aba-470f-b2ee-fcdf596983ae" Jul 28 13:47:52.597707 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:52.597676 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vvwjc" Jul 28 13:47:53.471964 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:53.471722 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khglj" Jul 28 13:47:53.472103 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:53.471764 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:47:53.472103 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:53.472007 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khglj" podUID="7b621445-7c29-4e1f-a55e-bc7161202598" Jul 28 13:47:53.472103 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:53.472061 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dlx" podUID="f5efacaa-1da7-4ed7-8bfa-2e805507fde7" Jul 28 13:47:53.601345 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:53.601310 2550 generic.go:356] "Generic (PLEG): container finished" podID="51d01c0c-2588-432a-bce0-03459112ffb9" containerID="4f8caa961913a3596e8eb25055288c2b39d6c6b20b5ba10fceb00f0c0c5b2fa9" exitCode=0 Jul 28 13:47:53.602207 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:53.601385 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8br7q" event={"ID":"51d01c0c-2588-432a-bce0-03459112ffb9","Type":"ContainerDied","Data":"4f8caa961913a3596e8eb25055288c2b39d6c6b20b5ba10fceb00f0c0c5b2fa9"} Jul 28 13:47:53.603685 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:53.603649 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wktfs" event={"ID":"a390d1e3-916f-43f8-b75f-940d222744a8","Type":"ContainerStarted","Data":"f085bc5583cb1387722562dfefb3e25bac886c4e76573cf80dc862023e12cea7"} Jul 28 13:47:53.606948 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:53.606931 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4nb4j_f498bb4f-ae84-4a84-ba7d-189d4a713911/ovn-acl-logging/0.log" Jul 28 13:47:53.607332 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:53.607311 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" event={"ID":"f498bb4f-ae84-4a84-ba7d-189d4a713911","Type":"ContainerStarted","Data":"15fdb355b574c67931a31e3d6d1fc2fc52bfcb1ca6a659bc3d74f5027c829be1"} Jul 28 13:47:53.607655 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:53.607634 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:53.607739 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:53.607712 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:53.607798 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:53.607755 2550 scope.go:117] "RemoveContainer" containerID="4218457c1b4e91612607ab1b7c11ef8a386c71a16c37ec23512d9f247541a015" Jul 28 13:47:53.607798 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:53.607762 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:53.624076 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:53.624057 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:53.625572 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:53.625555 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:47:53.678589 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:53.678518 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wktfs" podStartSLOduration=3.986058892 podStartE2EDuration="27.678502891s" totalImagesPullingTime="23.692443999s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 13:47:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:47:53.677996344 +0000 UTC m=+27.813193236" watchObservedRunningTime="2026-07-28 13:47:53.678502891 +0000 UTC m=+27.813699782" Jul 28 13:47:54.472339 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:54.472257 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z2sg" Jul 28 13:47:54.472649 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:54.472369 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z2sg" podUID="fbc15680-5aba-470f-b2ee-fcdf596983ae" Jul 28 13:47:54.613066 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:54.613037 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4nb4j_f498bb4f-ae84-4a84-ba7d-189d4a713911/ovn-acl-logging/0.log" Jul 28 13:47:54.613486 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:54.613426 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" event={"ID":"f498bb4f-ae84-4a84-ba7d-189d4a713911","Type":"ContainerStarted","Data":"482e9eecbfd482455855ae89cbf86d7bd737cf3895a06e9691c028c92e44e1cf"} Jul 28 13:47:54.645758 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:54.645707 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" podStartSLOduration=9.296807266 podStartE2EDuration="28.645693607s" totalImagesPullingTime="19.348886341s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 13:47:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:47:54.645633738 +0000 UTC m=+28.780830632" watchObservedRunningTime="2026-07-28 13:47:54.645693607 +0000 UTC m=+28.780890500" Jul 28 13:47:55.471735 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:55.471703 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khglj" Jul 28 13:47:55.471907 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:55.471703 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:47:55.471907 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:55.471821 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khglj" podUID="7b621445-7c29-4e1f-a55e-bc7161202598" Jul 28 13:47:55.471907 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:55.471881 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dlx" podUID="f5efacaa-1da7-4ed7-8bfa-2e805507fde7" Jul 28 13:47:56.472920 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:56.472734 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z2sg" Jul 28 13:47:56.473303 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:56.472974 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z2sg" podUID="fbc15680-5aba-470f-b2ee-fcdf596983ae" Jul 28 13:47:56.619670 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:56.619632 2550 generic.go:356] "Generic (PLEG): container finished" podID="51d01c0c-2588-432a-bce0-03459112ffb9" containerID="39e8c7b666618ea5cc0539c804ad63d651081d92eda32658a59cd83a0ff3b90b" exitCode=0 Jul 28 13:47:56.619831 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:56.619682 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8br7q" event={"ID":"51d01c0c-2588-432a-bce0-03459112ffb9","Type":"ContainerDied","Data":"39e8c7b666618ea5cc0539c804ad63d651081d92eda32658a59cd83a0ff3b90b"} Jul 28 13:47:57.472175 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:57.472138 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:47:57.472349 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:57.472251 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dlx" podUID="f5efacaa-1da7-4ed7-8bfa-2e805507fde7" Jul 28 13:47:57.472349 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:57.472306 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khglj" Jul 28 13:47:57.472451 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:57.472423 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khglj" podUID="7b621445-7c29-4e1f-a55e-bc7161202598" Jul 28 13:47:58.471720 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:58.471692 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z2sg" Jul 28 13:47:58.471980 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:58.471829 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z2sg" podUID="fbc15680-5aba-470f-b2ee-fcdf596983ae" Jul 28 13:47:58.625607 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:58.625528 2550 generic.go:356] "Generic (PLEG): container finished" podID="51d01c0c-2588-432a-bce0-03459112ffb9" containerID="f89af9951de124c97a4561faece0be418f20cc732fad86978914d7f1169b669a" exitCode=0 Jul 28 13:47:58.625607 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:58.625569 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8br7q" event={"ID":"51d01c0c-2588-432a-bce0-03459112ffb9","Type":"ContainerDied","Data":"f89af9951de124c97a4561faece0be418f20cc732fad86978914d7f1169b669a"} Jul 28 13:47:59.128796 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:59.128759 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f5efacaa-1da7-4ed7-8bfa-2e805507fde7-original-pull-secret\") pod \"global-pull-secret-syncer-89dlx\" (UID: \"f5efacaa-1da7-4ed7-8bfa-2e805507fde7\") " pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:47:59.128949 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:59.128806 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b621445-7c29-4e1f-a55e-bc7161202598-metrics-certs\") pod \"network-metrics-daemon-khglj\" (UID: \"7b621445-7c29-4e1f-a55e-bc7161202598\") " pod="openshift-multus/network-metrics-daemon-khglj" Jul 28 13:47:59.128949 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:59.128943 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 13:47:59.129016 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:59.129008 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5efacaa-1da7-4ed7-8bfa-2e805507fde7-original-pull-secret podName:f5efacaa-1da7-4ed7-8bfa-2e805507fde7 nodeName:}" failed. No retries permitted until 2026-07-28 13:48:31.128991206 +0000 UTC m=+65.264188082 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f5efacaa-1da7-4ed7-8bfa-2e805507fde7-original-pull-secret") pod "global-pull-secret-syncer-89dlx" (UID: "f5efacaa-1da7-4ed7-8bfa-2e805507fde7") : object "kube-system"/"original-pull-secret" not registered Jul 28 13:47:59.129058 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:59.129031 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 13:47:59.129097 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:59.129084 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b621445-7c29-4e1f-a55e-bc7161202598-metrics-certs podName:7b621445-7c29-4e1f-a55e-bc7161202598 nodeName:}" failed. No retries permitted until 2026-07-28 13:48:31.129071258 +0000 UTC m=+65.264268132 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b621445-7c29-4e1f-a55e-bc7161202598-metrics-certs") pod "network-metrics-daemon-khglj" (UID: "7b621445-7c29-4e1f-a55e-bc7161202598") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 13:47:59.230174 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:59.230135 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzjvh\" (UniqueName: \"kubernetes.io/projected/fbc15680-5aba-470f-b2ee-fcdf596983ae-kube-api-access-fzjvh\") pod \"network-check-target-2z2sg\" (UID: \"fbc15680-5aba-470f-b2ee-fcdf596983ae\") " pod="openshift-network-diagnostics/network-check-target-2z2sg" Jul 28 13:47:59.230383 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:59.230284 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 13:47:59.230383 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:59.230307 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 13:47:59.230383 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:59.230316 2550 projected.go:196] Error preparing data for projected volume kube-api-access-fzjvh for pod openshift-network-diagnostics/network-check-target-2z2sg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 13:47:59.230383 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:59.230382 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fbc15680-5aba-470f-b2ee-fcdf596983ae-kube-api-access-fzjvh podName:fbc15680-5aba-470f-b2ee-fcdf596983ae nodeName:}" failed. No retries permitted until 2026-07-28 13:48:31.230361389 +0000 UTC m=+65.365558276 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-fzjvh" (UniqueName: "kubernetes.io/projected/fbc15680-5aba-470f-b2ee-fcdf596983ae-kube-api-access-fzjvh") pod "network-check-target-2z2sg" (UID: "fbc15680-5aba-470f-b2ee-fcdf596983ae") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 13:47:59.472103 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:59.472022 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:47:59.472103 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:47:59.472043 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khglj" Jul 28 13:47:59.472548 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:59.472155 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dlx" podUID="f5efacaa-1da7-4ed7-8bfa-2e805507fde7" Jul 28 13:47:59.472548 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:47:59.472290 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khglj" podUID="7b621445-7c29-4e1f-a55e-bc7161202598" Jul 28 13:48:00.472665 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:00.472622 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z2sg" Jul 28 13:48:00.473120 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:48:00.472766 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z2sg" podUID="fbc15680-5aba-470f-b2ee-fcdf596983ae" Jul 28 13:48:01.472575 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:01.472535 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khglj" Jul 28 13:48:01.472575 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:01.472580 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:48:01.473108 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:48:01.472665 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khglj" podUID="7b621445-7c29-4e1f-a55e-bc7161202598" Jul 28 13:48:01.473108 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:48:01.472796 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dlx" podUID="f5efacaa-1da7-4ed7-8bfa-2e805507fde7" Jul 28 13:48:02.472523 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:02.472484 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z2sg" Jul 28 13:48:02.472712 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:48:02.472617 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z2sg" podUID="fbc15680-5aba-470f-b2ee-fcdf596983ae" Jul 28 13:48:03.472332 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:03.472291 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khglj" Jul 28 13:48:03.472765 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:48:03.472461 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khglj" podUID="7b621445-7c29-4e1f-a55e-bc7161202598" Jul 28 13:48:03.472765 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:03.472530 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:48:03.472765 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:48:03.472640 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dlx" podUID="f5efacaa-1da7-4ed7-8bfa-2e805507fde7" Jul 28 13:48:03.641736 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:03.641698 2550 generic.go:356] "Generic (PLEG): container finished" podID="51d01c0c-2588-432a-bce0-03459112ffb9" containerID="d592140f1ca99d7d5aef654e110e421b778fe4c8130ffa6f1acf8cc7a4a01364" exitCode=0 Jul 28 13:48:03.641885 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:03.641747 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8br7q" event={"ID":"51d01c0c-2588-432a-bce0-03459112ffb9","Type":"ContainerDied","Data":"d592140f1ca99d7d5aef654e110e421b778fe4c8130ffa6f1acf8cc7a4a01364"} Jul 28 13:48:04.278009 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:04.277960 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-89dlx"] Jul 28 13:48:04.278176 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:04.278158 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:48:04.278335 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:48:04.278301 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dlx" podUID="f5efacaa-1da7-4ed7-8bfa-2e805507fde7" Jul 28 13:48:04.278867 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:04.278811 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2z2sg"] Jul 28 13:48:04.278979 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:04.278914 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z2sg" Jul 28 13:48:04.279038 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:48:04.279002 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z2sg" podUID="fbc15680-5aba-470f-b2ee-fcdf596983ae" Jul 28 13:48:04.279512 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:04.279480 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-khglj"] Jul 28 13:48:04.280034 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:04.279582 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khglj" Jul 28 13:48:04.280034 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:48:04.279694 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khglj" podUID="7b621445-7c29-4e1f-a55e-bc7161202598" Jul 28 13:48:04.648278 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:04.648193 2550 generic.go:356] "Generic (PLEG): container finished" podID="51d01c0c-2588-432a-bce0-03459112ffb9" containerID="70ad115581c152632e3ea3c32e0a00416c9e67018926befc4fb3af8441c12599" exitCode=0 Jul 28 13:48:04.648278 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:04.648238 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8br7q" event={"ID":"51d01c0c-2588-432a-bce0-03459112ffb9","Type":"ContainerDied","Data":"70ad115581c152632e3ea3c32e0a00416c9e67018926befc4fb3af8441c12599"} Jul 28 13:48:05.472235 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:05.472159 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z2sg" Jul 28 13:48:05.472362 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:48:05.472254 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z2sg" podUID="fbc15680-5aba-470f-b2ee-fcdf596983ae" Jul 28 13:48:05.653217 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:05.653181 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8br7q" event={"ID":"51d01c0c-2588-432a-bce0-03459112ffb9","Type":"ContainerStarted","Data":"a165bdcef309989a8dc3ed60617a5420941c55e37233663b086ed99c86138396"} Jul 28 13:48:05.679734 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:05.679672 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-8br7q" podStartSLOduration=8.109583012 podStartE2EDuration="39.679649547s" totalImagesPullingTime="31.570066535s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 13:47:26 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:48:05.679553038 +0000 UTC m=+39.814749932" watchObservedRunningTime="2026-07-28 13:48:05.679649547 +0000 UTC m=+39.814846446" Jul 28 13:48:06.473361 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:06.473268 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:48:06.473542 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:48:06.473381 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dlx" podUID="f5efacaa-1da7-4ed7-8bfa-2e805507fde7" Jul 28 13:48:06.473542 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:06.473431 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khglj" Jul 28 13:48:06.473621 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:48:06.473562 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khglj" podUID="7b621445-7c29-4e1f-a55e-bc7161202598" Jul 28 13:48:07.472497 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.472460 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z2sg" Jul 28 13:48:07.472948 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:48:07.472557 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z2sg" podUID="fbc15680-5aba-470f-b2ee-fcdf596983ae" Jul 28 13:48:07.692617 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.692538 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-63.ec2.internal" event="NodeReady" Jul 28 13:48:07.692755 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.692669 2550 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 28 13:48:07.749615 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.749587 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-qb9g2"] Jul 28 13:48:07.780328 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.780301 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7q4gj"] Jul 28 13:48:07.780489 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.780445 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qb9g2" Jul 28 13:48:07.784256 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.784232 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 28 13:48:07.784417 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.784246 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 28 13:48:07.784417 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.784232 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xhnmn\"" Jul 28 13:48:07.795996 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.795973 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qb9g2"] Jul 28 13:48:07.796152 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.796015 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7q4gj"] Jul 28 13:48:07.796152 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.796038 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-qk8tc"] Jul 28 13:48:07.796152 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.796049 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7q4gj" Jul 28 13:48:07.798910 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.798856 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 28 13:48:07.799055 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.798949 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2j595\"" Jul 28 13:48:07.799055 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.798967 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 28 13:48:07.799457 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.799439 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 28 13:48:07.818155 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.818127 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qk8tc"] Jul 28 13:48:07.818276 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.818236 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qk8tc" Jul 28 13:48:07.820955 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.820934 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bhc89\"" Jul 28 13:48:07.820955 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.820949 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 28 13:48:07.821174 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.820983 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 28 13:48:07.821215 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.821204 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 28 13:48:07.821378 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.821366 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 28 13:48:07.894332 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.894295 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/404a2af9-c2b5-49c9-ad60-f7bb293afd8a-crio-socket\") pod \"insights-runtime-extractor-qk8tc\" (UID: \"404a2af9-c2b5-49c9-ad60-f7bb293afd8a\") " pod="openshift-insights/insights-runtime-extractor-qk8tc" Jul 28 13:48:07.894332 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.894331 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rjjg\" (UniqueName: \"kubernetes.io/projected/779de240-4570-4d0d-b38b-d5e64c01ab73-kube-api-access-8rjjg\") pod \"ingress-canary-7q4gj\" (UID: \"779de240-4570-4d0d-b38b-d5e64c01ab73\") " pod="openshift-ingress-canary/ingress-canary-7q4gj" Jul 28 13:48:07.894604 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.894358 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/30b7a4cf-a585-4366-bab1-28e9037be498-metrics-tls\") pod \"dns-default-qb9g2\" (UID: \"30b7a4cf-a585-4366-bab1-28e9037be498\") " pod="openshift-dns/dns-default-qb9g2" Jul 28 13:48:07.894604 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.894374 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30b7a4cf-a585-4366-bab1-28e9037be498-tmp-dir\") pod \"dns-default-qb9g2\" (UID: \"30b7a4cf-a585-4366-bab1-28e9037be498\") " pod="openshift-dns/dns-default-qb9g2" Jul 28 13:48:07.894604 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.894447 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9l4g8\" (UniqueName: \"kubernetes.io/projected/30b7a4cf-a585-4366-bab1-28e9037be498-kube-api-access-9l4g8\") pod \"dns-default-qb9g2\" (UID: \"30b7a4cf-a585-4366-bab1-28e9037be498\") " pod="openshift-dns/dns-default-qb9g2" Jul 28 13:48:07.894604 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.894530 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/404a2af9-c2b5-49c9-ad60-f7bb293afd8a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qk8tc\" (UID: \"404a2af9-c2b5-49c9-ad60-f7bb293afd8a\") " pod="openshift-insights/insights-runtime-extractor-qk8tc" Jul 28 13:48:07.894604 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.894569 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/404a2af9-c2b5-49c9-ad60-f7bb293afd8a-data-volume\") pod \"insights-runtime-extractor-qk8tc\" (UID: \"404a2af9-c2b5-49c9-ad60-f7bb293afd8a\") " pod="openshift-insights/insights-runtime-extractor-qk8tc" Jul 28 13:48:07.894604 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.894592 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/30b7a4cf-a585-4366-bab1-28e9037be498-config-volume\") pod \"dns-default-qb9g2\" (UID: \"30b7a4cf-a585-4366-bab1-28e9037be498\") " pod="openshift-dns/dns-default-qb9g2" Jul 28 13:48:07.894805 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.894619 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/404a2af9-c2b5-49c9-ad60-f7bb293afd8a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qk8tc\" (UID: \"404a2af9-c2b5-49c9-ad60-f7bb293afd8a\") " pod="openshift-insights/insights-runtime-extractor-qk8tc" Jul 28 13:48:07.894805 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.894683 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/779de240-4570-4d0d-b38b-d5e64c01ab73-cert\") pod \"ingress-canary-7q4gj\" (UID: \"779de240-4570-4d0d-b38b-d5e64c01ab73\") " pod="openshift-ingress-canary/ingress-canary-7q4gj" Jul 28 13:48:07.894805 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.894723 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v7rz\" (UniqueName: \"kubernetes.io/projected/404a2af9-c2b5-49c9-ad60-f7bb293afd8a-kube-api-access-4v7rz\") pod \"insights-runtime-extractor-qk8tc\" (UID: \"404a2af9-c2b5-49c9-ad60-f7bb293afd8a\") " pod="openshift-insights/insights-runtime-extractor-qk8tc" Jul 28 13:48:07.995793 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.995696 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/404a2af9-c2b5-49c9-ad60-f7bb293afd8a-crio-socket\") pod \"insights-runtime-extractor-qk8tc\" (UID: \"404a2af9-c2b5-49c9-ad60-f7bb293afd8a\") " pod="openshift-insights/insights-runtime-extractor-qk8tc" Jul 28 13:48:07.995793 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.995741 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rjjg\" (UniqueName: \"kubernetes.io/projected/779de240-4570-4d0d-b38b-d5e64c01ab73-kube-api-access-8rjjg\") pod \"ingress-canary-7q4gj\" (UID: \"779de240-4570-4d0d-b38b-d5e64c01ab73\") " pod="openshift-ingress-canary/ingress-canary-7q4gj" Jul 28 13:48:07.995793 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.995766 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/30b7a4cf-a585-4366-bab1-28e9037be498-metrics-tls\") pod \"dns-default-qb9g2\" (UID: \"30b7a4cf-a585-4366-bab1-28e9037be498\") " pod="openshift-dns/dns-default-qb9g2" Jul 28 13:48:07.995793 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.995788 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30b7a4cf-a585-4366-bab1-28e9037be498-tmp-dir\") pod \"dns-default-qb9g2\" (UID: \"30b7a4cf-a585-4366-bab1-28e9037be498\") " pod="openshift-dns/dns-default-qb9g2" Jul 28 13:48:07.996096 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.995903 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l4g8\" (UniqueName: \"kubernetes.io/projected/30b7a4cf-a585-4366-bab1-28e9037be498-kube-api-access-9l4g8\") pod \"dns-default-qb9g2\" (UID: \"30b7a4cf-a585-4366-bab1-28e9037be498\") " pod="openshift-dns/dns-default-qb9g2" Jul 28 13:48:07.996096 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.995947 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/404a2af9-c2b5-49c9-ad60-f7bb293afd8a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qk8tc\" (UID: \"404a2af9-c2b5-49c9-ad60-f7bb293afd8a\") " pod="openshift-insights/insights-runtime-extractor-qk8tc" Jul 28 13:48:07.996096 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.995904 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/404a2af9-c2b5-49c9-ad60-f7bb293afd8a-crio-socket\") pod \"insights-runtime-extractor-qk8tc\" (UID: \"404a2af9-c2b5-49c9-ad60-f7bb293afd8a\") " pod="openshift-insights/insights-runtime-extractor-qk8tc" Jul 28 13:48:07.996096 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.996056 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/404a2af9-c2b5-49c9-ad60-f7bb293afd8a-data-volume\") pod \"insights-runtime-extractor-qk8tc\" (UID: \"404a2af9-c2b5-49c9-ad60-f7bb293afd8a\") " pod="openshift-insights/insights-runtime-extractor-qk8tc" Jul 28 13:48:07.996096 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.996086 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/30b7a4cf-a585-4366-bab1-28e9037be498-config-volume\") pod \"dns-default-qb9g2\" (UID: \"30b7a4cf-a585-4366-bab1-28e9037be498\") " pod="openshift-dns/dns-default-qb9g2" Jul 28 13:48:07.996337 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.996115 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/404a2af9-c2b5-49c9-ad60-f7bb293afd8a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qk8tc\" (UID: \"404a2af9-c2b5-49c9-ad60-f7bb293afd8a\") " pod="openshift-insights/insights-runtime-extractor-qk8tc" Jul 28 13:48:07.996337 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.996131 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30b7a4cf-a585-4366-bab1-28e9037be498-tmp-dir\") pod \"dns-default-qb9g2\" (UID: \"30b7a4cf-a585-4366-bab1-28e9037be498\") " pod="openshift-dns/dns-default-qb9g2" Jul 28 13:48:07.996337 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.996260 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/779de240-4570-4d0d-b38b-d5e64c01ab73-cert\") pod \"ingress-canary-7q4gj\" (UID: \"779de240-4570-4d0d-b38b-d5e64c01ab73\") " pod="openshift-ingress-canary/ingress-canary-7q4gj" Jul 28 13:48:07.996337 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.996320 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v7rz\" (UniqueName: \"kubernetes.io/projected/404a2af9-c2b5-49c9-ad60-f7bb293afd8a-kube-api-access-4v7rz\") pod \"insights-runtime-extractor-qk8tc\" (UID: \"404a2af9-c2b5-49c9-ad60-f7bb293afd8a\") " pod="openshift-insights/insights-runtime-extractor-qk8tc" Jul 28 13:48:07.996549 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.996461 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/404a2af9-c2b5-49c9-ad60-f7bb293afd8a-data-volume\") pod \"insights-runtime-extractor-qk8tc\" (UID: \"404a2af9-c2b5-49c9-ad60-f7bb293afd8a\") " pod="openshift-insights/insights-runtime-extractor-qk8tc" Jul 28 13:48:07.996710 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.996692 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/30b7a4cf-a585-4366-bab1-28e9037be498-config-volume\") pod \"dns-default-qb9g2\" (UID: \"30b7a4cf-a585-4366-bab1-28e9037be498\") " pod="openshift-dns/dns-default-qb9g2" Jul 28 13:48:07.996770 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.996717 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/404a2af9-c2b5-49c9-ad60-f7bb293afd8a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qk8tc\" (UID: \"404a2af9-c2b5-49c9-ad60-f7bb293afd8a\") " pod="openshift-insights/insights-runtime-extractor-qk8tc" Jul 28 13:48:07.999894 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.999866 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/30b7a4cf-a585-4366-bab1-28e9037be498-metrics-tls\") pod \"dns-default-qb9g2\" (UID: \"30b7a4cf-a585-4366-bab1-28e9037be498\") " pod="openshift-dns/dns-default-qb9g2" Jul 28 13:48:07.999989 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.999901 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/404a2af9-c2b5-49c9-ad60-f7bb293afd8a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qk8tc\" (UID: \"404a2af9-c2b5-49c9-ad60-f7bb293afd8a\") " pod="openshift-insights/insights-runtime-extractor-qk8tc" Jul 28 13:48:07.999989 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:07.999925 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/779de240-4570-4d0d-b38b-d5e64c01ab73-cert\") pod \"ingress-canary-7q4gj\" (UID: \"779de240-4570-4d0d-b38b-d5e64c01ab73\") " pod="openshift-ingress-canary/ingress-canary-7q4gj" Jul 28 13:48:08.006353 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:08.006332 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9l4g8\" (UniqueName: \"kubernetes.io/projected/30b7a4cf-a585-4366-bab1-28e9037be498-kube-api-access-9l4g8\") pod \"dns-default-qb9g2\" (UID: \"30b7a4cf-a585-4366-bab1-28e9037be498\") " pod="openshift-dns/dns-default-qb9g2" Jul 28 13:48:08.006787 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:08.006765 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v7rz\" (UniqueName: \"kubernetes.io/projected/404a2af9-c2b5-49c9-ad60-f7bb293afd8a-kube-api-access-4v7rz\") pod \"insights-runtime-extractor-qk8tc\" (UID: \"404a2af9-c2b5-49c9-ad60-f7bb293afd8a\") " pod="openshift-insights/insights-runtime-extractor-qk8tc" Jul 28 13:48:08.006847 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:08.006826 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rjjg\" (UniqueName: \"kubernetes.io/projected/779de240-4570-4d0d-b38b-d5e64c01ab73-kube-api-access-8rjjg\") pod \"ingress-canary-7q4gj\" (UID: \"779de240-4570-4d0d-b38b-d5e64c01ab73\") " pod="openshift-ingress-canary/ingress-canary-7q4gj" Jul 28 13:48:08.090666 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:08.090622 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qb9g2" Jul 28 13:48:08.104901 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:08.104873 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7q4gj" Jul 28 13:48:08.126919 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:08.126895 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qk8tc" Jul 28 13:48:08.308060 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:08.307872 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qb9g2"] Jul 28 13:48:08.310701 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:08.310660 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7q4gj"] Jul 28 13:48:08.311451 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:48:08.311424 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30b7a4cf_a585_4366_bab1_28e9037be498.slice/crio-a595cce8857774afde3b8ce96c853599aa3ffee1037f2a8ac17f8a215f3629a9 WatchSource:0}: Error finding container a595cce8857774afde3b8ce96c853599aa3ffee1037f2a8ac17f8a215f3629a9: Status 404 returned error can't find the container with id a595cce8857774afde3b8ce96c853599aa3ffee1037f2a8ac17f8a215f3629a9 Jul 28 13:48:08.314436 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:48:08.314405 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod779de240_4570_4d0d_b38b_d5e64c01ab73.slice/crio-d4d6af459c72e1e57c67eb9f9642a102c63400fec61ce5d24898f75d956110b0 WatchSource:0}: Error finding container d4d6af459c72e1e57c67eb9f9642a102c63400fec61ce5d24898f75d956110b0: Status 404 returned error can't find the container with id d4d6af459c72e1e57c67eb9f9642a102c63400fec61ce5d24898f75d956110b0 Jul 28 13:48:08.325195 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:08.325174 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qk8tc"] Jul 28 13:48:08.327505 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:48:08.327483 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod404a2af9_c2b5_49c9_ad60_f7bb293afd8a.slice/crio-8c78a9188a9be27b991245057046439176b38ec02ee17e62ddf74f559c4b9c8a WatchSource:0}: Error finding container 8c78a9188a9be27b991245057046439176b38ec02ee17e62ddf74f559c4b9c8a: Status 404 returned error can't find the container with id 8c78a9188a9be27b991245057046439176b38ec02ee17e62ddf74f559c4b9c8a Jul 28 13:48:08.471912 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:08.471875 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khglj" Jul 28 13:48:08.472072 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:08.471876 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:48:08.474570 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:08.474544 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 28 13:48:08.475009 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:08.474628 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 28 13:48:08.475009 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:08.474628 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zrb86\"" Jul 28 13:48:08.661183 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:08.661097 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qb9g2" event={"ID":"30b7a4cf-a585-4366-bab1-28e9037be498","Type":"ContainerStarted","Data":"a595cce8857774afde3b8ce96c853599aa3ffee1037f2a8ac17f8a215f3629a9"} Jul 28 13:48:08.662209 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:08.662188 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qk8tc" event={"ID":"404a2af9-c2b5-49c9-ad60-f7bb293afd8a","Type":"ContainerStarted","Data":"8c78a9188a9be27b991245057046439176b38ec02ee17e62ddf74f559c4b9c8a"} Jul 28 13:48:08.663163 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:08.663143 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7q4gj" event={"ID":"779de240-4570-4d0d-b38b-d5e64c01ab73","Type":"ContainerStarted","Data":"d4d6af459c72e1e57c67eb9f9642a102c63400fec61ce5d24898f75d956110b0"} Jul 28 13:48:08.956117 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:08.956026 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-bdqn6"] Jul 28 13:48:08.964869 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:08.964839 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-bdqn6" Jul 28 13:48:08.967077 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:08.967053 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-bdqn6"] Jul 28 13:48:08.967731 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:08.967694 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-pwv8m\"" Jul 28 13:48:08.967843 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:08.967790 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 28 13:48:09.104813 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:09.104776 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4c634ef8-d53e-4282-b7b7-699f55ed22a2-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-bdqn6\" (UID: \"4c634ef8-d53e-4282-b7b7-699f55ed22a2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-bdqn6" Jul 28 13:48:09.206234 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:09.206112 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4c634ef8-d53e-4282-b7b7-699f55ed22a2-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-bdqn6\" (UID: \"4c634ef8-d53e-4282-b7b7-699f55ed22a2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-bdqn6" Jul 28 13:48:09.206464 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:48:09.206262 2550 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 28 13:48:09.206464 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:48:09.206334 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c634ef8-d53e-4282-b7b7-699f55ed22a2-tls-certificates podName:4c634ef8-d53e-4282-b7b7-699f55ed22a2 nodeName:}" failed. No retries permitted until 2026-07-28 13:48:09.706313337 +0000 UTC m=+43.841510210 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/4c634ef8-d53e-4282-b7b7-699f55ed22a2-tls-certificates") pod "prometheus-operator-admission-webhook-7f8d78f54b-bdqn6" (UID: "4c634ef8-d53e-4282-b7b7-699f55ed22a2") : secret "prometheus-operator-admission-webhook-tls" not found Jul 28 13:48:09.472767 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:09.472688 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z2sg" Jul 28 13:48:09.476182 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:09.475900 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 28 13:48:09.476182 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:09.476152 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-crbr2\"" Jul 28 13:48:09.478443 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:09.477049 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 28 13:48:09.711715 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:09.711630 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4c634ef8-d53e-4282-b7b7-699f55ed22a2-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-bdqn6\" (UID: \"4c634ef8-d53e-4282-b7b7-699f55ed22a2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-bdqn6" Jul 28 13:48:09.716548 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:09.716523 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4c634ef8-d53e-4282-b7b7-699f55ed22a2-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-bdqn6\" (UID: \"4c634ef8-d53e-4282-b7b7-699f55ed22a2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-bdqn6" Jul 28 13:48:09.876928 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:09.876779 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-bdqn6" Jul 28 13:48:10.495810 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:10.495778 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-bdqn6"] Jul 28 13:48:10.499775 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:48:10.499744 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c634ef8_d53e_4282_b7b7_699f55ed22a2.slice/crio-85bcb953bbd20aa43fbb34a75483523f67c36d2e49692bb02f79e4e7f76e5ba5 WatchSource:0}: Error finding container 85bcb953bbd20aa43fbb34a75483523f67c36d2e49692bb02f79e4e7f76e5ba5: Status 404 returned error can't find the container with id 85bcb953bbd20aa43fbb34a75483523f67c36d2e49692bb02f79e4e7f76e5ba5 Jul 28 13:48:10.670838 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:10.670792 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-bdqn6" event={"ID":"4c634ef8-d53e-4282-b7b7-699f55ed22a2","Type":"ContainerStarted","Data":"85bcb953bbd20aa43fbb34a75483523f67c36d2e49692bb02f79e4e7f76e5ba5"} Jul 28 13:48:11.675223 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:11.675028 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qb9g2" event={"ID":"30b7a4cf-a585-4366-bab1-28e9037be498","Type":"ContainerStarted","Data":"08be92650de5f02f24cf8ec358f996921b0319030e820fe8c65875fb8346091b"} Jul 28 13:48:15.687060 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:15.687021 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qb9g2" event={"ID":"30b7a4cf-a585-4366-bab1-28e9037be498","Type":"ContainerStarted","Data":"790c41a9c5a989e744c8d39259a3321b553693a503684d4125a17f397cfcdb43"} Jul 28 13:48:15.687934 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:15.687625 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-qb9g2" Jul 28 13:48:15.688622 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:15.688597 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7q4gj" event={"ID":"779de240-4570-4d0d-b38b-d5e64c01ab73","Type":"ContainerStarted","Data":"2acf408a64ad3268f08ecc8aef313a28513f466e41370618a5386741fe1de013"} Jul 28 13:48:15.707378 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:15.707328 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-qb9g2" podStartSLOduration=6.338300964 podStartE2EDuration="8.707310876s" totalImagesPullingTime="2.369009912s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:48:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:48:15.707108592 +0000 UTC m=+49.842305486" watchObservedRunningTime="2026-07-28 13:48:15.707310876 +0000 UTC m=+49.842507768" Jul 28 13:48:15.727019 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:15.726974 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7q4gj" podStartSLOduration=2.029623207 podStartE2EDuration="8.726947661s" totalImagesPullingTime="6.697324454s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:48:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:48:15.726066485 +0000 UTC m=+49.861263374" watchObservedRunningTime="2026-07-28 13:48:15.726947661 +0000 UTC m=+49.862144532" Jul 28 13:48:16.695516 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:16.695290 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qk8tc" event={"ID":"404a2af9-c2b5-49c9-ad60-f7bb293afd8a","Type":"ContainerStarted","Data":"ddcb28589c92bd9e963e7852c6e0f9359c18732b30540c5cb3832e60aa1084a6"} Jul 28 13:48:16.697457 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:16.697426 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-bdqn6" event={"ID":"4c634ef8-d53e-4282-b7b7-699f55ed22a2","Type":"ContainerStarted","Data":"8880fb72f495b336e407e2e168d346c48f3bea07e877c5cbba49bf765008db90"} Jul 28 13:48:16.698165 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:16.697914 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-bdqn6" Jul 28 13:48:16.704956 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:16.704904 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-bdqn6" Jul 28 13:48:16.715775 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:16.715717 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-bdqn6" podStartSLOduration=3.33967654 podStartE2EDuration="8.715696499s" totalImagesPullingTime="5.376019959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:48:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:48:16.715067739 +0000 UTC m=+50.850264633" watchObservedRunningTime="2026-07-28 13:48:16.715696499 +0000 UTC m=+50.850893372" Jul 28 13:48:17.057999 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:17.057910 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-26vgk"] Jul 28 13:48:17.070698 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:17.070669 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-26vgk" Jul 28 13:48:17.073856 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:17.073831 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 28 13:48:17.073973 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:17.073857 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 28 13:48:17.074672 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:17.074653 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 28 13:48:17.074768 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:17.074717 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 28 13:48:17.074831 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:17.074717 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 28 13:48:17.078226 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:17.078209 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-rbj4j\"" Jul 28 13:48:17.080995 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:17.080972 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-26vgk"] Jul 28 13:48:17.174426 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:17.174372 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1fe2993-d45f-4585-8c28-fd5b0780bf31-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-26vgk\" (UID: \"a1fe2993-d45f-4585-8c28-fd5b0780bf31\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-26vgk" Jul 28 13:48:17.174627 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:17.174438 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a1fe2993-d45f-4585-8c28-fd5b0780bf31-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-26vgk\" (UID: \"a1fe2993-d45f-4585-8c28-fd5b0780bf31\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-26vgk" Jul 28 13:48:17.174627 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:17.174457 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bttbd\" (UniqueName: \"kubernetes.io/projected/a1fe2993-d45f-4585-8c28-fd5b0780bf31-kube-api-access-bttbd\") pod \"prometheus-operator-84d69f794d-26vgk\" (UID: \"a1fe2993-d45f-4585-8c28-fd5b0780bf31\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-26vgk" Jul 28 13:48:17.174627 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:17.174498 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a1fe2993-d45f-4585-8c28-fd5b0780bf31-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-26vgk\" (UID: \"a1fe2993-d45f-4585-8c28-fd5b0780bf31\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-26vgk" Jul 28 13:48:17.275851 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:17.275816 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a1fe2993-d45f-4585-8c28-fd5b0780bf31-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-26vgk\" (UID: \"a1fe2993-d45f-4585-8c28-fd5b0780bf31\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-26vgk" Jul 28 13:48:17.276010 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:17.275902 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1fe2993-d45f-4585-8c28-fd5b0780bf31-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-26vgk\" (UID: \"a1fe2993-d45f-4585-8c28-fd5b0780bf31\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-26vgk" Jul 28 13:48:17.276010 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:17.275923 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a1fe2993-d45f-4585-8c28-fd5b0780bf31-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-26vgk\" (UID: \"a1fe2993-d45f-4585-8c28-fd5b0780bf31\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-26vgk" Jul 28 13:48:17.276010 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:17.275939 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bttbd\" (UniqueName: \"kubernetes.io/projected/a1fe2993-d45f-4585-8c28-fd5b0780bf31-kube-api-access-bttbd\") pod \"prometheus-operator-84d69f794d-26vgk\" (UID: \"a1fe2993-d45f-4585-8c28-fd5b0780bf31\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-26vgk" Jul 28 13:48:17.276127 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:48:17.276038 2550 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 28 13:48:17.276127 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:48:17.276119 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1fe2993-d45f-4585-8c28-fd5b0780bf31-prometheus-operator-tls podName:a1fe2993-d45f-4585-8c28-fd5b0780bf31 nodeName:}" failed. No retries permitted until 2026-07-28 13:48:17.776095042 +0000 UTC m=+51.911291913 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/a1fe2993-d45f-4585-8c28-fd5b0780bf31-prometheus-operator-tls") pod "prometheus-operator-84d69f794d-26vgk" (UID: "a1fe2993-d45f-4585-8c28-fd5b0780bf31") : secret "prometheus-operator-tls" not found Jul 28 13:48:17.276592 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:17.276576 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a1fe2993-d45f-4585-8c28-fd5b0780bf31-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-26vgk\" (UID: \"a1fe2993-d45f-4585-8c28-fd5b0780bf31\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-26vgk" Jul 28 13:48:17.279980 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:17.279961 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a1fe2993-d45f-4585-8c28-fd5b0780bf31-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-26vgk\" (UID: \"a1fe2993-d45f-4585-8c28-fd5b0780bf31\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-26vgk" Jul 28 13:48:17.285946 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:17.285918 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bttbd\" (UniqueName: \"kubernetes.io/projected/a1fe2993-d45f-4585-8c28-fd5b0780bf31-kube-api-access-bttbd\") pod \"prometheus-operator-84d69f794d-26vgk\" (UID: \"a1fe2993-d45f-4585-8c28-fd5b0780bf31\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-26vgk" Jul 28 13:48:17.780710 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:17.780674 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1fe2993-d45f-4585-8c28-fd5b0780bf31-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-26vgk\" (UID: \"a1fe2993-d45f-4585-8c28-fd5b0780bf31\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-26vgk" Jul 28 13:48:17.783279 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:17.783253 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1fe2993-d45f-4585-8c28-fd5b0780bf31-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-26vgk\" (UID: \"a1fe2993-d45f-4585-8c28-fd5b0780bf31\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-26vgk" Jul 28 13:48:17.981955 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:17.981915 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-26vgk" Jul 28 13:48:19.678114 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:19.678082 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-26vgk"] Jul 28 13:48:19.716998 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:48:19.716964 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1fe2993_d45f_4585_8c28_fd5b0780bf31.slice/crio-8d0290ec2d5d18651e15f90e8dd5894e6a9a719a0abd6f06fa0af96a9f797e7b WatchSource:0}: Error finding container 8d0290ec2d5d18651e15f90e8dd5894e6a9a719a0abd6f06fa0af96a9f797e7b: Status 404 returned error can't find the container with id 8d0290ec2d5d18651e15f90e8dd5894e6a9a719a0abd6f06fa0af96a9f797e7b Jul 28 13:48:20.713077 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:20.712790 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-26vgk" event={"ID":"a1fe2993-d45f-4585-8c28-fd5b0780bf31","Type":"ContainerStarted","Data":"8d0290ec2d5d18651e15f90e8dd5894e6a9a719a0abd6f06fa0af96a9f797e7b"} Jul 28 13:48:20.715635 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:20.715566 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qk8tc" event={"ID":"404a2af9-c2b5-49c9-ad60-f7bb293afd8a","Type":"ContainerStarted","Data":"12b707044949cb3efdf2662c3528c2994793ee4771aea8c3b04e74260c723c05"} Jul 28 13:48:21.722277 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:21.722238 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-26vgk" event={"ID":"a1fe2993-d45f-4585-8c28-fd5b0780bf31","Type":"ContainerStarted","Data":"f360882f27f90aef8a121a1077fb4edca2979876eaa99eb19ec44d43781d0b99"} Jul 28 13:48:21.722961 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:21.722304 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-26vgk" event={"ID":"a1fe2993-d45f-4585-8c28-fd5b0780bf31","Type":"ContainerStarted","Data":"6b7cf34c130a35d92cb4a1ac039bbeaa8939e84b2d9d9ad45ec6d7310faeadc4"} Jul 28 13:48:21.742049 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:21.741989 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-26vgk" podStartSLOduration=3.501130665 podStartE2EDuration="4.741967279s" totalImagesPullingTime="1.240836614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:48:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:48:21.740828803 +0000 UTC m=+55.876025696" watchObservedRunningTime="2026-07-28 13:48:21.741967279 +0000 UTC m=+55.877164171" Jul 28 13:48:22.700922 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:22.700892 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-qb9g2" Jul 28 13:48:23.459115 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.459081 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbclx"] Jul 28 13:48:23.461536 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.461513 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbclx" Jul 28 13:48:23.464048 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.464030 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 28 13:48:23.464425 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.464355 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 28 13:48:23.464658 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.464644 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-qqx6f\"" Jul 28 13:48:23.470747 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.470725 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-5m62f"] Jul 28 13:48:23.472657 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.472643 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-5m62f" Jul 28 13:48:23.475157 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.475131 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 28 13:48:23.475373 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.475210 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbclx"] Jul 28 13:48:23.475612 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.475595 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 28 13:48:23.476149 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.476123 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-kmjwk\"" Jul 28 13:48:23.476634 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.476608 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 28 13:48:23.492339 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.492317 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-5m62f"] Jul 28 13:48:23.515022 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.515001 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-x4rhq"] Jul 28 13:48:23.517500 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.517480 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-x4rhq" Jul 28 13:48:23.520011 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.519938 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 28 13:48:23.520448 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.520177 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 28 13:48:23.520448 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.520231 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bqk49\"" Jul 28 13:48:23.520603 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.520473 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 28 13:48:23.528887 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.528868 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bc5f9125-f382-4871-92f6-fdd66afeafd3-node-exporter-wtmp\") pod \"node-exporter-x4rhq\" (UID: \"bc5f9125-f382-4871-92f6-fdd66afeafd3\") " pod="openshift-monitoring/node-exporter-x4rhq" Jul 28 13:48:23.528989 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.528897 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bc5f9125-f382-4871-92f6-fdd66afeafd3-node-exporter-textfile\") pod \"node-exporter-x4rhq\" (UID: \"bc5f9125-f382-4871-92f6-fdd66afeafd3\") " pod="openshift-monitoring/node-exporter-x4rhq" Jul 28 13:48:23.528989 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.528922 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bc5f9125-f382-4871-92f6-fdd66afeafd3-metrics-client-ca\") pod \"node-exporter-x4rhq\" (UID: \"bc5f9125-f382-4871-92f6-fdd66afeafd3\") " pod="openshift-monitoring/node-exporter-x4rhq" Jul 28 13:48:23.528989 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.528948 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/14a143a8-6917-4413-8d3b-333cb12e6f7b-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-5m62f\" (UID: \"14a143a8-6917-4413-8d3b-333cb12e6f7b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-5m62f" Jul 28 13:48:23.528989 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.528968 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/14a143a8-6917-4413-8d3b-333cb12e6f7b-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-5m62f\" (UID: \"14a143a8-6917-4413-8d3b-333cb12e6f7b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-5m62f" Jul 28 13:48:23.529233 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.529057 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/14a143a8-6917-4413-8d3b-333cb12e6f7b-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-5m62f\" (UID: \"14a143a8-6917-4413-8d3b-333cb12e6f7b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-5m62f" Jul 28 13:48:23.529233 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.529095 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bc5f9125-f382-4871-92f6-fdd66afeafd3-sys\") pod \"node-exporter-x4rhq\" (UID: \"bc5f9125-f382-4871-92f6-fdd66afeafd3\") " pod="openshift-monitoring/node-exporter-x4rhq" Jul 28 13:48:23.529233 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.529143 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bc5f9125-f382-4871-92f6-fdd66afeafd3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-x4rhq\" (UID: \"bc5f9125-f382-4871-92f6-fdd66afeafd3\") " pod="openshift-monitoring/node-exporter-x4rhq" Jul 28 13:48:23.529233 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.529198 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bc5f9125-f382-4871-92f6-fdd66afeafd3-node-exporter-accelerators-collector-config\") pod \"node-exporter-x4rhq\" (UID: \"bc5f9125-f382-4871-92f6-fdd66afeafd3\") " pod="openshift-monitoring/node-exporter-x4rhq" Jul 28 13:48:23.529233 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.529228 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bc5f9125-f382-4871-92f6-fdd66afeafd3-root\") pod \"node-exporter-x4rhq\" (UID: \"bc5f9125-f382-4871-92f6-fdd66afeafd3\") " pod="openshift-monitoring/node-exporter-x4rhq" Jul 28 13:48:23.529496 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.529259 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmwjc\" (UniqueName: \"kubernetes.io/projected/8daae3b7-94f7-47e0-98dd-567bde1a0038-kube-api-access-xmwjc\") pod \"openshift-state-metrics-b8f49f5c6-cbclx\" (UID: \"8daae3b7-94f7-47e0-98dd-567bde1a0038\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbclx" Jul 28 13:48:23.529496 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.529362 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8daae3b7-94f7-47e0-98dd-567bde1a0038-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-cbclx\" (UID: \"8daae3b7-94f7-47e0-98dd-567bde1a0038\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbclx" Jul 28 13:48:23.529496 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.529417 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/14a143a8-6917-4413-8d3b-333cb12e6f7b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-5m62f\" (UID: \"14a143a8-6917-4413-8d3b-333cb12e6f7b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-5m62f" Jul 28 13:48:23.529496 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.529444 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8bsf\" (UniqueName: \"kubernetes.io/projected/bc5f9125-f382-4871-92f6-fdd66afeafd3-kube-api-access-n8bsf\") pod \"node-exporter-x4rhq\" (UID: \"bc5f9125-f382-4871-92f6-fdd66afeafd3\") " pod="openshift-monitoring/node-exporter-x4rhq" Jul 28 13:48:23.529496 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.529489 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8daae3b7-94f7-47e0-98dd-567bde1a0038-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-cbclx\" (UID: \"8daae3b7-94f7-47e0-98dd-567bde1a0038\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbclx" Jul 28 13:48:23.529723 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.529520 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/14a143a8-6917-4413-8d3b-333cb12e6f7b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-5m62f\" (UID: \"14a143a8-6917-4413-8d3b-333cb12e6f7b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-5m62f" Jul 28 13:48:23.529723 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.529546 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnmdn\" (UniqueName: \"kubernetes.io/projected/14a143a8-6917-4413-8d3b-333cb12e6f7b-kube-api-access-rnmdn\") pod \"kube-state-metrics-56d9f944b4-5m62f\" (UID: \"14a143a8-6917-4413-8d3b-333cb12e6f7b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-5m62f" Jul 28 13:48:23.529723 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.529570 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bc5f9125-f382-4871-92f6-fdd66afeafd3-node-exporter-tls\") pod \"node-exporter-x4rhq\" (UID: \"bc5f9125-f382-4871-92f6-fdd66afeafd3\") " pod="openshift-monitoring/node-exporter-x4rhq" Jul 28 13:48:23.529723 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.529633 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8daae3b7-94f7-47e0-98dd-567bde1a0038-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-cbclx\" (UID: \"8daae3b7-94f7-47e0-98dd-567bde1a0038\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbclx" Jul 28 13:48:23.630630 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.630597 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bc5f9125-f382-4871-92f6-fdd66afeafd3-metrics-client-ca\") pod \"node-exporter-x4rhq\" (UID: \"bc5f9125-f382-4871-92f6-fdd66afeafd3\") " pod="openshift-monitoring/node-exporter-x4rhq" Jul 28 13:48:23.630630 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.630631 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/14a143a8-6917-4413-8d3b-333cb12e6f7b-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-5m62f\" (UID: \"14a143a8-6917-4413-8d3b-333cb12e6f7b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-5m62f" Jul 28 13:48:23.630866 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.630656 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/14a143a8-6917-4413-8d3b-333cb12e6f7b-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-5m62f\" (UID: \"14a143a8-6917-4413-8d3b-333cb12e6f7b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-5m62f" Jul 28 13:48:23.630866 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:48:23.630759 2550 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 28 13:48:23.630866 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.630782 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/14a143a8-6917-4413-8d3b-333cb12e6f7b-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-5m62f\" (UID: \"14a143a8-6917-4413-8d3b-333cb12e6f7b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-5m62f" Jul 28 13:48:23.630866 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.630816 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bc5f9125-f382-4871-92f6-fdd66afeafd3-sys\") pod \"node-exporter-x4rhq\" (UID: \"bc5f9125-f382-4871-92f6-fdd66afeafd3\") " pod="openshift-monitoring/node-exporter-x4rhq" Jul 28 13:48:23.630866 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.630833 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bc5f9125-f382-4871-92f6-fdd66afeafd3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-x4rhq\" (UID: \"bc5f9125-f382-4871-92f6-fdd66afeafd3\") " pod="openshift-monitoring/node-exporter-x4rhq" Jul 28 13:48:23.631103 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.630874 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bc5f9125-f382-4871-92f6-fdd66afeafd3-node-exporter-accelerators-collector-config\") pod \"node-exporter-x4rhq\" (UID: \"bc5f9125-f382-4871-92f6-fdd66afeafd3\") " pod="openshift-monitoring/node-exporter-x4rhq" Jul 28 13:48:23.631103 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.630893 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bc5f9125-f382-4871-92f6-fdd66afeafd3-sys\") pod \"node-exporter-x4rhq\" (UID: \"bc5f9125-f382-4871-92f6-fdd66afeafd3\") " pod="openshift-monitoring/node-exporter-x4rhq" Jul 28 13:48:23.631103 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:48:23.630899 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14a143a8-6917-4413-8d3b-333cb12e6f7b-kube-state-metrics-tls podName:14a143a8-6917-4413-8d3b-333cb12e6f7b nodeName:}" failed. No retries permitted until 2026-07-28 13:48:24.130874248 +0000 UTC m=+58.266071125 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/14a143a8-6917-4413-8d3b-333cb12e6f7b-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-5m62f" (UID: "14a143a8-6917-4413-8d3b-333cb12e6f7b") : secret "kube-state-metrics-tls" not found Jul 28 13:48:23.631103 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.630953 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bc5f9125-f382-4871-92f6-fdd66afeafd3-root\") pod \"node-exporter-x4rhq\" (UID: \"bc5f9125-f382-4871-92f6-fdd66afeafd3\") " pod="openshift-monitoring/node-exporter-x4rhq" Jul 28 13:48:23.631103 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.630986 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmwjc\" (UniqueName: \"kubernetes.io/projected/8daae3b7-94f7-47e0-98dd-567bde1a0038-kube-api-access-xmwjc\") pod \"openshift-state-metrics-b8f49f5c6-cbclx\" (UID: \"8daae3b7-94f7-47e0-98dd-567bde1a0038\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbclx" Jul 28 13:48:23.631103 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.631018 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8daae3b7-94f7-47e0-98dd-567bde1a0038-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-cbclx\" (UID: \"8daae3b7-94f7-47e0-98dd-567bde1a0038\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbclx" Jul 28 13:48:23.631103 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.631047 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/14a143a8-6917-4413-8d3b-333cb12e6f7b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-5m62f\" (UID: \"14a143a8-6917-4413-8d3b-333cb12e6f7b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-5m62f" Jul 28 13:48:23.631103 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.631055 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bc5f9125-f382-4871-92f6-fdd66afeafd3-root\") pod \"node-exporter-x4rhq\" (UID: \"bc5f9125-f382-4871-92f6-fdd66afeafd3\") " pod="openshift-monitoring/node-exporter-x4rhq" Jul 28 13:48:23.631103 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.631071 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8bsf\" (UniqueName: \"kubernetes.io/projected/bc5f9125-f382-4871-92f6-fdd66afeafd3-kube-api-access-n8bsf\") pod \"node-exporter-x4rhq\" (UID: \"bc5f9125-f382-4871-92f6-fdd66afeafd3\") " pod="openshift-monitoring/node-exporter-x4rhq" Jul 28 13:48:23.631103 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.631076 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/14a143a8-6917-4413-8d3b-333cb12e6f7b-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-5m62f\" (UID: \"14a143a8-6917-4413-8d3b-333cb12e6f7b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-5m62f" Jul 28 13:48:23.631614 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.631111 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8daae3b7-94f7-47e0-98dd-567bde1a0038-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-cbclx\" (UID: \"8daae3b7-94f7-47e0-98dd-567bde1a0038\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbclx" Jul 28 13:48:23.631614 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.631139 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/14a143a8-6917-4413-8d3b-333cb12e6f7b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-5m62f\" (UID: \"14a143a8-6917-4413-8d3b-333cb12e6f7b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-5m62f" Jul 28 13:48:23.631614 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.631166 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnmdn\" (UniqueName: \"kubernetes.io/projected/14a143a8-6917-4413-8d3b-333cb12e6f7b-kube-api-access-rnmdn\") pod \"kube-state-metrics-56d9f944b4-5m62f\" (UID: \"14a143a8-6917-4413-8d3b-333cb12e6f7b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-5m62f" Jul 28 13:48:23.631614 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.631186 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bc5f9125-f382-4871-92f6-fdd66afeafd3-node-exporter-tls\") pod \"node-exporter-x4rhq\" (UID: \"bc5f9125-f382-4871-92f6-fdd66afeafd3\") " pod="openshift-monitoring/node-exporter-x4rhq" Jul 28 13:48:23.631614 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.631214 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8daae3b7-94f7-47e0-98dd-567bde1a0038-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-cbclx\" (UID: \"8daae3b7-94f7-47e0-98dd-567bde1a0038\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbclx" Jul 28 13:48:23.631614 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.631242 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bc5f9125-f382-4871-92f6-fdd66afeafd3-node-exporter-wtmp\") pod \"node-exporter-x4rhq\" (UID: \"bc5f9125-f382-4871-92f6-fdd66afeafd3\") " pod="openshift-monitoring/node-exporter-x4rhq" Jul 28 13:48:23.631614 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.631254 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bc5f9125-f382-4871-92f6-fdd66afeafd3-metrics-client-ca\") pod \"node-exporter-x4rhq\" (UID: \"bc5f9125-f382-4871-92f6-fdd66afeafd3\") " pod="openshift-monitoring/node-exporter-x4rhq" Jul 28 13:48:23.631614 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.631268 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bc5f9125-f382-4871-92f6-fdd66afeafd3-node-exporter-textfile\") pod \"node-exporter-x4rhq\" (UID: \"bc5f9125-f382-4871-92f6-fdd66afeafd3\") " pod="openshift-monitoring/node-exporter-x4rhq" Jul 28 13:48:23.631614 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.631461 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bc5f9125-f382-4871-92f6-fdd66afeafd3-node-exporter-accelerators-collector-config\") pod \"node-exporter-x4rhq\" (UID: \"bc5f9125-f382-4871-92f6-fdd66afeafd3\") " pod="openshift-monitoring/node-exporter-x4rhq" Jul 28 13:48:23.631614 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:48:23.631551 2550 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 28 13:48:23.631614 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.631562 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bc5f9125-f382-4871-92f6-fdd66afeafd3-node-exporter-textfile\") pod \"node-exporter-x4rhq\" (UID: \"bc5f9125-f382-4871-92f6-fdd66afeafd3\") " pod="openshift-monitoring/node-exporter-x4rhq" Jul 28 13:48:23.631614 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.631573 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/14a143a8-6917-4413-8d3b-333cb12e6f7b-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-5m62f\" (UID: \"14a143a8-6917-4413-8d3b-333cb12e6f7b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-5m62f" Jul 28 13:48:23.631614 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:48:23.631596 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc5f9125-f382-4871-92f6-fdd66afeafd3-node-exporter-tls podName:bc5f9125-f382-4871-92f6-fdd66afeafd3 nodeName:}" failed. No retries permitted until 2026-07-28 13:48:24.131580816 +0000 UTC m=+58.266777699 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/bc5f9125-f382-4871-92f6-fdd66afeafd3-node-exporter-tls") pod "node-exporter-x4rhq" (UID: "bc5f9125-f382-4871-92f6-fdd66afeafd3") : secret "node-exporter-tls" not found Jul 28 13:48:23.632192 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.631741 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bc5f9125-f382-4871-92f6-fdd66afeafd3-node-exporter-wtmp\") pod \"node-exporter-x4rhq\" (UID: \"bc5f9125-f382-4871-92f6-fdd66afeafd3\") " pod="openshift-monitoring/node-exporter-x4rhq" Jul 28 13:48:23.632192 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.631794 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8daae3b7-94f7-47e0-98dd-567bde1a0038-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-cbclx\" (UID: \"8daae3b7-94f7-47e0-98dd-567bde1a0038\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbclx" Jul 28 13:48:23.632192 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:48:23.631819 2550 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 28 13:48:23.632192 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:48:23.631890 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8daae3b7-94f7-47e0-98dd-567bde1a0038-openshift-state-metrics-tls podName:8daae3b7-94f7-47e0-98dd-567bde1a0038 nodeName:}" failed. No retries permitted until 2026-07-28 13:48:24.131874985 +0000 UTC m=+58.267071872 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/8daae3b7-94f7-47e0-98dd-567bde1a0038-openshift-state-metrics-tls") pod "openshift-state-metrics-b8f49f5c6-cbclx" (UID: "8daae3b7-94f7-47e0-98dd-567bde1a0038") : secret "openshift-state-metrics-tls" not found Jul 28 13:48:23.632192 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.631961 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/14a143a8-6917-4413-8d3b-333cb12e6f7b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-5m62f\" (UID: \"14a143a8-6917-4413-8d3b-333cb12e6f7b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-5m62f" Jul 28 13:48:23.634443 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.634417 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bc5f9125-f382-4871-92f6-fdd66afeafd3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-x4rhq\" (UID: \"bc5f9125-f382-4871-92f6-fdd66afeafd3\") " pod="openshift-monitoring/node-exporter-x4rhq" Jul 28 13:48:23.634545 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.634473 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/14a143a8-6917-4413-8d3b-333cb12e6f7b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-5m62f\" (UID: \"14a143a8-6917-4413-8d3b-333cb12e6f7b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-5m62f" Jul 28 13:48:23.634545 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.634539 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8daae3b7-94f7-47e0-98dd-567bde1a0038-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-cbclx\" (UID: \"8daae3b7-94f7-47e0-98dd-567bde1a0038\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbclx" Jul 28 13:48:23.644217 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.644192 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8bsf\" (UniqueName: \"kubernetes.io/projected/bc5f9125-f382-4871-92f6-fdd66afeafd3-kube-api-access-n8bsf\") pod \"node-exporter-x4rhq\" (UID: \"bc5f9125-f382-4871-92f6-fdd66afeafd3\") " pod="openshift-monitoring/node-exporter-x4rhq" Jul 28 13:48:23.644484 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.644468 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmwjc\" (UniqueName: \"kubernetes.io/projected/8daae3b7-94f7-47e0-98dd-567bde1a0038-kube-api-access-xmwjc\") pod \"openshift-state-metrics-b8f49f5c6-cbclx\" (UID: \"8daae3b7-94f7-47e0-98dd-567bde1a0038\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbclx" Jul 28 13:48:23.644532 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.644516 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnmdn\" (UniqueName: \"kubernetes.io/projected/14a143a8-6917-4413-8d3b-333cb12e6f7b-kube-api-access-rnmdn\") pod \"kube-state-metrics-56d9f944b4-5m62f\" (UID: \"14a143a8-6917-4413-8d3b-333cb12e6f7b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-5m62f" Jul 28 13:48:23.732526 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.732440 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qk8tc" event={"ID":"404a2af9-c2b5-49c9-ad60-f7bb293afd8a","Type":"ContainerStarted","Data":"cba4d72da931a402bc1bf48321d14dc62e6357f63a0fd561f8360afabf6643b8"} Jul 28 13:48:23.751973 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:23.751923 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-qk8tc" podStartSLOduration=1.769706864 podStartE2EDuration="16.751907471s" totalImagesPullingTime="14.982200607s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 13:48:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:48:23.751580808 +0000 UTC m=+57.886777702" watchObservedRunningTime="2026-07-28 13:48:23.751907471 +0000 UTC m=+57.887104345" Jul 28 13:48:24.134580 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.134491 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8daae3b7-94f7-47e0-98dd-567bde1a0038-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-cbclx\" (UID: \"8daae3b7-94f7-47e0-98dd-567bde1a0038\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbclx" Jul 28 13:48:24.134580 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.134528 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bc5f9125-f382-4871-92f6-fdd66afeafd3-node-exporter-tls\") pod \"node-exporter-x4rhq\" (UID: \"bc5f9125-f382-4871-92f6-fdd66afeafd3\") " pod="openshift-monitoring/node-exporter-x4rhq" Jul 28 13:48:24.134580 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.134560 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/14a143a8-6917-4413-8d3b-333cb12e6f7b-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-5m62f\" (UID: \"14a143a8-6917-4413-8d3b-333cb12e6f7b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-5m62f" Jul 28 13:48:24.134821 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:48:24.134627 2550 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 28 13:48:24.134821 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:48:24.134643 2550 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 28 13:48:24.134821 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:48:24.134695 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8daae3b7-94f7-47e0-98dd-567bde1a0038-openshift-state-metrics-tls podName:8daae3b7-94f7-47e0-98dd-567bde1a0038 nodeName:}" failed. No retries permitted until 2026-07-28 13:48:25.134679676 +0000 UTC m=+59.269876547 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/8daae3b7-94f7-47e0-98dd-567bde1a0038-openshift-state-metrics-tls") pod "openshift-state-metrics-b8f49f5c6-cbclx" (UID: "8daae3b7-94f7-47e0-98dd-567bde1a0038") : secret "openshift-state-metrics-tls" not found Jul 28 13:48:24.134821 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:48:24.134709 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc5f9125-f382-4871-92f6-fdd66afeafd3-node-exporter-tls podName:bc5f9125-f382-4871-92f6-fdd66afeafd3 nodeName:}" failed. No retries permitted until 2026-07-28 13:48:25.134702008 +0000 UTC m=+59.269898878 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/bc5f9125-f382-4871-92f6-fdd66afeafd3-node-exporter-tls") pod "node-exporter-x4rhq" (UID: "bc5f9125-f382-4871-92f6-fdd66afeafd3") : secret "node-exporter-tls" not found Jul 28 13:48:24.136841 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.136823 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/14a143a8-6917-4413-8d3b-333cb12e6f7b-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-5m62f\" (UID: \"14a143a8-6917-4413-8d3b-333cb12e6f7b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-5m62f" Jul 28 13:48:24.385622 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.385533 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-5m62f" Jul 28 13:48:24.518846 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.518816 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-5m62f"] Jul 28 13:48:24.522056 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:48:24.522022 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14a143a8_6917_4413_8d3b_333cb12e6f7b.slice/crio-2489234a1120abcc9bcb7269379acc6b3d3ac9f42a4947e4357e1cf2355d235e WatchSource:0}: Error finding container 2489234a1120abcc9bcb7269379acc6b3d3ac9f42a4947e4357e1cf2355d235e: Status 404 returned error can't find the container with id 2489234a1120abcc9bcb7269379acc6b3d3ac9f42a4947e4357e1cf2355d235e Jul 28 13:48:24.522778 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.522698 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 13:48:24.526253 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.526234 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.528921 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.528892 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 28 13:48:24.529048 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.529018 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 28 13:48:24.529166 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.529150 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-mn46k\"" Jul 28 13:48:24.529234 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.529194 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 28 13:48:24.529234 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.529150 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 28 13:48:24.529387 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.529369 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 28 13:48:24.529486 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.529434 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 28 13:48:24.529486 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.529371 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 28 13:48:24.529486 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.529371 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 28 13:48:24.529621 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.529538 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 28 13:48:24.537047 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.537025 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.537159 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.537051 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-web-config\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.537159 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.537134 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.537230 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.537166 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d43548cb-241a-490a-a94f-7c08efe068cc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.537230 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.537185 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d43548cb-241a-490a-a94f-7c08efe068cc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.537295 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.537264 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.537336 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.537308 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.537336 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.537328 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d43548cb-241a-490a-a94f-7c08efe068cc-config-out\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.537407 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.537345 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mn8nb\" (UniqueName: \"kubernetes.io/projected/d43548cb-241a-490a-a94f-7c08efe068cc-kube-api-access-mn8nb\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.537448 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.537424 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d43548cb-241a-490a-a94f-7c08efe068cc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.537485 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.537461 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d43548cb-241a-490a-a94f-7c08efe068cc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.537519 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.537509 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-config-volume\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.537550 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.537534 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.544811 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.544784 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 13:48:24.638064 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.637976 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d43548cb-241a-490a-a94f-7c08efe068cc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.638064 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.638013 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d43548cb-241a-490a-a94f-7c08efe068cc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.638064 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.638037 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-config-volume\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.638064 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.638057 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.638373 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.638077 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.638373 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.638093 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-web-config\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.638373 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.638263 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.638373 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.638298 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d43548cb-241a-490a-a94f-7c08efe068cc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.638373 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.638321 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d43548cb-241a-490a-a94f-7c08efe068cc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.638744 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.638379 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.638744 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.638459 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.638744 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.638502 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d43548cb-241a-490a-a94f-7c08efe068cc-config-out\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.638744 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.638526 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mn8nb\" (UniqueName: \"kubernetes.io/projected/d43548cb-241a-490a-a94f-7c08efe068cc-kube-api-access-mn8nb\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.639023 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.638996 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d43548cb-241a-490a-a94f-7c08efe068cc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.639715 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.639375 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d43548cb-241a-490a-a94f-7c08efe068cc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.639715 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:48:24.639676 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d43548cb-241a-490a-a94f-7c08efe068cc-alertmanager-trusted-ca-bundle podName:d43548cb-241a-490a-a94f-7c08efe068cc nodeName:}" failed. No retries permitted until 2026-07-28 13:48:25.139653705 +0000 UTC m=+59.274850577 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/d43548cb-241a-490a-a94f-7c08efe068cc-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "d43548cb-241a-490a-a94f-7c08efe068cc") : configmap references non-existent config key: ca-bundle.crt Jul 28 13:48:24.641427 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.641307 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-config-volume\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.641427 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.641411 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.642006 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.641698 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d43548cb-241a-490a-a94f-7c08efe068cc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.642006 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.641968 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-web-config\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.642158 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.642095 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.642158 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.642135 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.642359 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.642336 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.642509 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.642491 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.642788 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.642774 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d43548cb-241a-490a-a94f-7c08efe068cc-config-out\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.648788 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.648764 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mn8nb\" (UniqueName: \"kubernetes.io/projected/d43548cb-241a-490a-a94f-7c08efe068cc-kube-api-access-mn8nb\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:24.736373 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:24.736339 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-5m62f" event={"ID":"14a143a8-6917-4413-8d3b-333cb12e6f7b","Type":"ContainerStarted","Data":"2489234a1120abcc9bcb7269379acc6b3d3ac9f42a4947e4357e1cf2355d235e"} Jul 28 13:48:25.141629 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.141593 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8daae3b7-94f7-47e0-98dd-567bde1a0038-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-cbclx\" (UID: \"8daae3b7-94f7-47e0-98dd-567bde1a0038\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbclx" Jul 28 13:48:25.141629 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.141632 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bc5f9125-f382-4871-92f6-fdd66afeafd3-node-exporter-tls\") pod \"node-exporter-x4rhq\" (UID: \"bc5f9125-f382-4871-92f6-fdd66afeafd3\") " pod="openshift-monitoring/node-exporter-x4rhq" Jul 28 13:48:25.141893 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.141654 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d43548cb-241a-490a-a94f-7c08efe068cc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:25.142466 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.142446 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d43548cb-241a-490a-a94f-7c08efe068cc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:25.144021 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.144000 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bc5f9125-f382-4871-92f6-fdd66afeafd3-node-exporter-tls\") pod \"node-exporter-x4rhq\" (UID: \"bc5f9125-f382-4871-92f6-fdd66afeafd3\") " pod="openshift-monitoring/node-exporter-x4rhq" Jul 28 13:48:25.144109 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.144026 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8daae3b7-94f7-47e0-98dd-567bde1a0038-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-cbclx\" (UID: \"8daae3b7-94f7-47e0-98dd-567bde1a0038\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbclx" Jul 28 13:48:25.270773 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.270735 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbclx" Jul 28 13:48:25.330055 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.330029 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-x4rhq" Jul 28 13:48:25.340950 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:48:25.340923 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc5f9125_f382_4871_92f6_fdd66afeafd3.slice/crio-fbc5d2d7764ed2e41afbb44c1254764413e831732a566131780b274f1c9b43aa WatchSource:0}: Error finding container fbc5d2d7764ed2e41afbb44c1254764413e831732a566131780b274f1c9b43aa: Status 404 returned error can't find the container with id fbc5d2d7764ed2e41afbb44c1254764413e831732a566131780b274f1c9b43aa Jul 28 13:48:25.396348 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.396314 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbclx"] Jul 28 13:48:25.398659 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:48:25.398630 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8daae3b7_94f7_47e0_98dd_567bde1a0038.slice/crio-8d5bd09654a60a072097c5fe0814df47b234054dd325542f0c82a91ec61f4cae WatchSource:0}: Error finding container 8d5bd09654a60a072097c5fe0814df47b234054dd325542f0c82a91ec61f4cae: Status 404 returned error can't find the container with id 8d5bd09654a60a072097c5fe0814df47b234054dd325542f0c82a91ec61f4cae Jul 28 13:48:25.435988 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.435966 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:48:25.553254 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.553186 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-ff6496dd8-k76hx"] Jul 28 13:48:25.559117 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.559092 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-ff6496dd8-k76hx" Jul 28 13:48:25.562212 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.562187 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 28 13:48:25.563022 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.562521 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 28 13:48:25.563022 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.562543 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 28 13:48:25.563022 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.562692 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 28 13:48:25.563022 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.562760 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-2rr07t0hlhe0i\"" Jul 28 13:48:25.563022 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.562871 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-twlqq\"" Jul 28 13:48:25.563022 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.562975 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 28 13:48:25.572089 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.572063 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-ff6496dd8-k76hx"] Jul 28 13:48:25.601544 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.601512 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 13:48:25.604908 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:48:25.604884 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd43548cb_241a_490a_a94f_7c08efe068cc.slice/crio-6d41498fb5f3f490a64bb54ddbeb0e3c1f59bcfa21b297253c3ae8b06bf40dfc WatchSource:0}: Error finding container 6d41498fb5f3f490a64bb54ddbeb0e3c1f59bcfa21b297253c3ae8b06bf40dfc: Status 404 returned error can't find the container with id 6d41498fb5f3f490a64bb54ddbeb0e3c1f59bcfa21b297253c3ae8b06bf40dfc Jul 28 13:48:25.628446 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.628417 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4nb4j" Jul 28 13:48:25.646810 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.646726 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4b4c841c-be47-457a-96d4-c2bef7851671-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-ff6496dd8-k76hx\" (UID: \"4b4c841c-be47-457a-96d4-c2bef7851671\") " pod="openshift-monitoring/thanos-querier-ff6496dd8-k76hx" Jul 28 13:48:25.646810 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.646772 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b4c841c-be47-457a-96d4-c2bef7851671-metrics-client-ca\") pod \"thanos-querier-ff6496dd8-k76hx\" (UID: \"4b4c841c-be47-457a-96d4-c2bef7851671\") " pod="openshift-monitoring/thanos-querier-ff6496dd8-k76hx" Jul 28 13:48:25.647023 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.646854 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4b4c841c-be47-457a-96d4-c2bef7851671-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-ff6496dd8-k76hx\" (UID: \"4b4c841c-be47-457a-96d4-c2bef7851671\") " pod="openshift-monitoring/thanos-querier-ff6496dd8-k76hx" Jul 28 13:48:25.647023 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.646964 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4b4c841c-be47-457a-96d4-c2bef7851671-secret-thanos-querier-tls\") pod \"thanos-querier-ff6496dd8-k76hx\" (UID: \"4b4c841c-be47-457a-96d4-c2bef7851671\") " pod="openshift-monitoring/thanos-querier-ff6496dd8-k76hx" Jul 28 13:48:25.647023 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.646994 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrxmq\" (UniqueName: \"kubernetes.io/projected/4b4c841c-be47-457a-96d4-c2bef7851671-kube-api-access-xrxmq\") pod \"thanos-querier-ff6496dd8-k76hx\" (UID: \"4b4c841c-be47-457a-96d4-c2bef7851671\") " pod="openshift-monitoring/thanos-querier-ff6496dd8-k76hx" Jul 28 13:48:25.647174 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.647088 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4b4c841c-be47-457a-96d4-c2bef7851671-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-ff6496dd8-k76hx\" (UID: \"4b4c841c-be47-457a-96d4-c2bef7851671\") " pod="openshift-monitoring/thanos-querier-ff6496dd8-k76hx" Jul 28 13:48:25.647174 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.647158 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4b4c841c-be47-457a-96d4-c2bef7851671-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-ff6496dd8-k76hx\" (UID: \"4b4c841c-be47-457a-96d4-c2bef7851671\") " pod="openshift-monitoring/thanos-querier-ff6496dd8-k76hx" Jul 28 13:48:25.647274 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.647184 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4b4c841c-be47-457a-96d4-c2bef7851671-secret-grpc-tls\") pod \"thanos-querier-ff6496dd8-k76hx\" (UID: \"4b4c841c-be47-457a-96d4-c2bef7851671\") " pod="openshift-monitoring/thanos-querier-ff6496dd8-k76hx" Jul 28 13:48:25.741531 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.741493 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d43548cb-241a-490a-a94f-7c08efe068cc","Type":"ContainerStarted","Data":"6d41498fb5f3f490a64bb54ddbeb0e3c1f59bcfa21b297253c3ae8b06bf40dfc"} Jul 28 13:48:25.743605 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.743572 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbclx" event={"ID":"8daae3b7-94f7-47e0-98dd-567bde1a0038","Type":"ContainerStarted","Data":"c1216f4357596166589894e35f5050f2dabcd049fd361966d20b7f1b9f6d6c8b"} Jul 28 13:48:25.743751 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.743627 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbclx" event={"ID":"8daae3b7-94f7-47e0-98dd-567bde1a0038","Type":"ContainerStarted","Data":"fac56926d4e069fe0612f95f319e61e8a57785e89ae2b88bdf6bc7d6a8889c13"} Jul 28 13:48:25.743751 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.743651 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbclx" event={"ID":"8daae3b7-94f7-47e0-98dd-567bde1a0038","Type":"ContainerStarted","Data":"8d5bd09654a60a072097c5fe0814df47b234054dd325542f0c82a91ec61f4cae"} Jul 28 13:48:25.745021 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.744946 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x4rhq" event={"ID":"bc5f9125-f382-4871-92f6-fdd66afeafd3","Type":"ContainerStarted","Data":"fbc5d2d7764ed2e41afbb44c1254764413e831732a566131780b274f1c9b43aa"} Jul 28 13:48:25.748442 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.748412 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4b4c841c-be47-457a-96d4-c2bef7851671-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-ff6496dd8-k76hx\" (UID: \"4b4c841c-be47-457a-96d4-c2bef7851671\") " pod="openshift-monitoring/thanos-querier-ff6496dd8-k76hx" Jul 28 13:48:25.748537 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.748490 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4b4c841c-be47-457a-96d4-c2bef7851671-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-ff6496dd8-k76hx\" (UID: \"4b4c841c-be47-457a-96d4-c2bef7851671\") " pod="openshift-monitoring/thanos-querier-ff6496dd8-k76hx" Jul 28 13:48:25.748537 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.748519 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4b4c841c-be47-457a-96d4-c2bef7851671-secret-grpc-tls\") pod \"thanos-querier-ff6496dd8-k76hx\" (UID: \"4b4c841c-be47-457a-96d4-c2bef7851671\") " pod="openshift-monitoring/thanos-querier-ff6496dd8-k76hx" Jul 28 13:48:25.748650 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.748566 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4b4c841c-be47-457a-96d4-c2bef7851671-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-ff6496dd8-k76hx\" (UID: \"4b4c841c-be47-457a-96d4-c2bef7851671\") " pod="openshift-monitoring/thanos-querier-ff6496dd8-k76hx" Jul 28 13:48:25.748766 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.748643 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b4c841c-be47-457a-96d4-c2bef7851671-metrics-client-ca\") pod \"thanos-querier-ff6496dd8-k76hx\" (UID: \"4b4c841c-be47-457a-96d4-c2bef7851671\") " pod="openshift-monitoring/thanos-querier-ff6496dd8-k76hx" Jul 28 13:48:25.748766 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.748698 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4b4c841c-be47-457a-96d4-c2bef7851671-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-ff6496dd8-k76hx\" (UID: \"4b4c841c-be47-457a-96d4-c2bef7851671\") " pod="openshift-monitoring/thanos-querier-ff6496dd8-k76hx" Jul 28 13:48:25.748766 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.748764 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4b4c841c-be47-457a-96d4-c2bef7851671-secret-thanos-querier-tls\") pod \"thanos-querier-ff6496dd8-k76hx\" (UID: \"4b4c841c-be47-457a-96d4-c2bef7851671\") " pod="openshift-monitoring/thanos-querier-ff6496dd8-k76hx" Jul 28 13:48:25.748766 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.748790 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrxmq\" (UniqueName: \"kubernetes.io/projected/4b4c841c-be47-457a-96d4-c2bef7851671-kube-api-access-xrxmq\") pod \"thanos-querier-ff6496dd8-k76hx\" (UID: \"4b4c841c-be47-457a-96d4-c2bef7851671\") " pod="openshift-monitoring/thanos-querier-ff6496dd8-k76hx" Jul 28 13:48:25.749587 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.749560 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b4c841c-be47-457a-96d4-c2bef7851671-metrics-client-ca\") pod \"thanos-querier-ff6496dd8-k76hx\" (UID: \"4b4c841c-be47-457a-96d4-c2bef7851671\") " pod="openshift-monitoring/thanos-querier-ff6496dd8-k76hx" Jul 28 13:48:25.751562 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.751518 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4b4c841c-be47-457a-96d4-c2bef7851671-secret-grpc-tls\") pod \"thanos-querier-ff6496dd8-k76hx\" (UID: \"4b4c841c-be47-457a-96d4-c2bef7851671\") " pod="openshift-monitoring/thanos-querier-ff6496dd8-k76hx" Jul 28 13:48:25.751664 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.751629 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4b4c841c-be47-457a-96d4-c2bef7851671-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-ff6496dd8-k76hx\" (UID: \"4b4c841c-be47-457a-96d4-c2bef7851671\") " pod="openshift-monitoring/thanos-querier-ff6496dd8-k76hx" Jul 28 13:48:25.751794 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.751768 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4b4c841c-be47-457a-96d4-c2bef7851671-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-ff6496dd8-k76hx\" (UID: \"4b4c841c-be47-457a-96d4-c2bef7851671\") " pod="openshift-monitoring/thanos-querier-ff6496dd8-k76hx" Jul 28 13:48:25.752134 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.752108 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4b4c841c-be47-457a-96d4-c2bef7851671-secret-thanos-querier-tls\") pod \"thanos-querier-ff6496dd8-k76hx\" (UID: \"4b4c841c-be47-457a-96d4-c2bef7851671\") " pod="openshift-monitoring/thanos-querier-ff6496dd8-k76hx" Jul 28 13:48:25.752680 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.752655 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4b4c841c-be47-457a-96d4-c2bef7851671-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-ff6496dd8-k76hx\" (UID: \"4b4c841c-be47-457a-96d4-c2bef7851671\") " pod="openshift-monitoring/thanos-querier-ff6496dd8-k76hx" Jul 28 13:48:25.752807 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.752787 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4b4c841c-be47-457a-96d4-c2bef7851671-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-ff6496dd8-k76hx\" (UID: \"4b4c841c-be47-457a-96d4-c2bef7851671\") " pod="openshift-monitoring/thanos-querier-ff6496dd8-k76hx" Jul 28 13:48:25.758201 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.758182 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrxmq\" (UniqueName: \"kubernetes.io/projected/4b4c841c-be47-457a-96d4-c2bef7851671-kube-api-access-xrxmq\") pod \"thanos-querier-ff6496dd8-k76hx\" (UID: \"4b4c841c-be47-457a-96d4-c2bef7851671\") " pod="openshift-monitoring/thanos-querier-ff6496dd8-k76hx" Jul 28 13:48:25.846234 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.846195 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-9ws9v"] Jul 28 13:48:25.850423 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.850376 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-9ws9v" Jul 28 13:48:25.853432 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.853382 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 28 13:48:25.853432 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.853450 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 28 13:48:25.853796 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.853749 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-f7fjg\"" Jul 28 13:48:25.858683 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.858629 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-9ws9v"] Jul 28 13:48:25.873209 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.873181 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-ff6496dd8-k76hx" Jul 28 13:48:25.950849 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:25.950811 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9kqp\" (UniqueName: \"kubernetes.io/projected/30347318-34fb-44a6-a955-9b3774e1f73c-kube-api-access-b9kqp\") pod \"downloads-7c97dc4555-9ws9v\" (UID: \"30347318-34fb-44a6-a955-9b3774e1f73c\") " pod="openshift-console/downloads-7c97dc4555-9ws9v" Jul 28 13:48:26.052304 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:26.052261 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9kqp\" (UniqueName: \"kubernetes.io/projected/30347318-34fb-44a6-a955-9b3774e1f73c-kube-api-access-b9kqp\") pod \"downloads-7c97dc4555-9ws9v\" (UID: \"30347318-34fb-44a6-a955-9b3774e1f73c\") " pod="openshift-console/downloads-7c97dc4555-9ws9v" Jul 28 13:48:26.065148 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:26.065113 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9kqp\" (UniqueName: \"kubernetes.io/projected/30347318-34fb-44a6-a955-9b3774e1f73c-kube-api-access-b9kqp\") pod \"downloads-7c97dc4555-9ws9v\" (UID: \"30347318-34fb-44a6-a955-9b3774e1f73c\") " pod="openshift-console/downloads-7c97dc4555-9ws9v" Jul 28 13:48:26.163347 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:26.163305 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-9ws9v" Jul 28 13:48:27.344260 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:27.344230 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-ff6496dd8-k76hx"] Jul 28 13:48:27.345473 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:48:27.345381 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b4c841c_be47_457a_96d4_c2bef7851671.slice/crio-c34eebafe8efc2452f8eafec0ca195ef0b5d42122d091213cc00577cbc1a9372 WatchSource:0}: Error finding container c34eebafe8efc2452f8eafec0ca195ef0b5d42122d091213cc00577cbc1a9372: Status 404 returned error can't find the container with id c34eebafe8efc2452f8eafec0ca195ef0b5d42122d091213cc00577cbc1a9372 Jul 28 13:48:27.356193 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:27.356172 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-9ws9v"] Jul 28 13:48:27.357742 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:48:27.357722 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30347318_34fb_44a6_a955_9b3774e1f73c.slice/crio-d8b3cebdc4e0c53025183d45f92cae75ba0a69bf44655a6202064364aeea22a4 WatchSource:0}: Error finding container d8b3cebdc4e0c53025183d45f92cae75ba0a69bf44655a6202064364aeea22a4: Status 404 returned error can't find the container with id d8b3cebdc4e0c53025183d45f92cae75ba0a69bf44655a6202064364aeea22a4 Jul 28 13:48:27.749196 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:27.749119 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6cfdb4569f-sxnmp"] Jul 28 13:48:27.752308 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:27.752286 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6cfdb4569f-sxnmp" Jul 28 13:48:27.755221 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:27.755197 2550 generic.go:356] "Generic (PLEG): container finished" podID="bc5f9125-f382-4871-92f6-fdd66afeafd3" containerID="5f9ff90d4c8e4603f114ca7917efe1498abe3fbf746962bf66d2bf7e873edba1" exitCode=0 Jul 28 13:48:27.755425 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:27.755256 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x4rhq" event={"ID":"bc5f9125-f382-4871-92f6-fdd66afeafd3","Type":"ContainerDied","Data":"5f9ff90d4c8e4603f114ca7917efe1498abe3fbf746962bf66d2bf7e873edba1"} Jul 28 13:48:27.756616 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:27.756593 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 28 13:48:27.756720 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:27.756613 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-qsgtc\"" Jul 28 13:48:27.756903 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:27.756608 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 28 13:48:27.757037 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:27.756882 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 28 13:48:27.757037 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:27.756889 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 28 13:48:27.757163 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:27.756905 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-eab04n8mb28bp\"" Jul 28 13:48:27.757712 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:27.757685 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-5m62f" event={"ID":"14a143a8-6917-4413-8d3b-333cb12e6f7b","Type":"ContainerStarted","Data":"e25e6a267f04597e4585ee56962e35f48c202b500a4b38e20e9df44603c4fcdd"} Jul 28 13:48:27.759115 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:27.759092 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-9ws9v" event={"ID":"30347318-34fb-44a6-a955-9b3774e1f73c","Type":"ContainerStarted","Data":"d8b3cebdc4e0c53025183d45f92cae75ba0a69bf44655a6202064364aeea22a4"} Jul 28 13:48:27.761102 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:27.761079 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-ff6496dd8-k76hx" event={"ID":"4b4c841c-be47-457a-96d4-c2bef7851671","Type":"ContainerStarted","Data":"c34eebafe8efc2452f8eafec0ca195ef0b5d42122d091213cc00577cbc1a9372"} Jul 28 13:48:27.765121 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:27.763595 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6cfdb4569f-sxnmp"] Jul 28 13:48:27.765680 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:27.765656 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/851f584e-e63a-469b-8043-194d66cce086-secret-metrics-server-client-certs\") pod \"metrics-server-6cfdb4569f-sxnmp\" (UID: \"851f584e-e63a-469b-8043-194d66cce086\") " pod="openshift-monitoring/metrics-server-6cfdb4569f-sxnmp" Jul 28 13:48:27.765765 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:27.765742 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/851f584e-e63a-469b-8043-194d66cce086-client-ca-bundle\") pod \"metrics-server-6cfdb4569f-sxnmp\" (UID: \"851f584e-e63a-469b-8043-194d66cce086\") " pod="openshift-monitoring/metrics-server-6cfdb4569f-sxnmp" Jul 28 13:48:27.765829 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:27.765817 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/851f584e-e63a-469b-8043-194d66cce086-audit-log\") pod \"metrics-server-6cfdb4569f-sxnmp\" (UID: \"851f584e-e63a-469b-8043-194d66cce086\") " pod="openshift-monitoring/metrics-server-6cfdb4569f-sxnmp" Jul 28 13:48:27.765883 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:27.765857 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/851f584e-e63a-469b-8043-194d66cce086-metrics-server-audit-profiles\") pod \"metrics-server-6cfdb4569f-sxnmp\" (UID: \"851f584e-e63a-469b-8043-194d66cce086\") " pod="openshift-monitoring/metrics-server-6cfdb4569f-sxnmp" Jul 28 13:48:27.765931 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:27.765910 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tlmj\" (UniqueName: \"kubernetes.io/projected/851f584e-e63a-469b-8043-194d66cce086-kube-api-access-8tlmj\") pod \"metrics-server-6cfdb4569f-sxnmp\" (UID: \"851f584e-e63a-469b-8043-194d66cce086\") " pod="openshift-monitoring/metrics-server-6cfdb4569f-sxnmp" Jul 28 13:48:27.766055 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:27.766005 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/851f584e-e63a-469b-8043-194d66cce086-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6cfdb4569f-sxnmp\" (UID: \"851f584e-e63a-469b-8043-194d66cce086\") " pod="openshift-monitoring/metrics-server-6cfdb4569f-sxnmp" Jul 28 13:48:27.766120 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:27.766100 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/851f584e-e63a-469b-8043-194d66cce086-secret-metrics-server-tls\") pod \"metrics-server-6cfdb4569f-sxnmp\" (UID: \"851f584e-e63a-469b-8043-194d66cce086\") " pod="openshift-monitoring/metrics-server-6cfdb4569f-sxnmp" Jul 28 13:48:27.867082 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:27.867023 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/851f584e-e63a-469b-8043-194d66cce086-client-ca-bundle\") pod \"metrics-server-6cfdb4569f-sxnmp\" (UID: \"851f584e-e63a-469b-8043-194d66cce086\") " pod="openshift-monitoring/metrics-server-6cfdb4569f-sxnmp" Jul 28 13:48:27.868338 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:27.867603 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/851f584e-e63a-469b-8043-194d66cce086-audit-log\") pod \"metrics-server-6cfdb4569f-sxnmp\" (UID: \"851f584e-e63a-469b-8043-194d66cce086\") " pod="openshift-monitoring/metrics-server-6cfdb4569f-sxnmp" Jul 28 13:48:27.868338 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:27.867655 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/851f584e-e63a-469b-8043-194d66cce086-metrics-server-audit-profiles\") pod \"metrics-server-6cfdb4569f-sxnmp\" (UID: \"851f584e-e63a-469b-8043-194d66cce086\") " pod="openshift-monitoring/metrics-server-6cfdb4569f-sxnmp" Jul 28 13:48:27.868338 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:27.867726 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tlmj\" (UniqueName: \"kubernetes.io/projected/851f584e-e63a-469b-8043-194d66cce086-kube-api-access-8tlmj\") pod \"metrics-server-6cfdb4569f-sxnmp\" (UID: \"851f584e-e63a-469b-8043-194d66cce086\") " pod="openshift-monitoring/metrics-server-6cfdb4569f-sxnmp" Jul 28 13:48:27.868338 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:27.867793 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/851f584e-e63a-469b-8043-194d66cce086-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6cfdb4569f-sxnmp\" (UID: \"851f584e-e63a-469b-8043-194d66cce086\") " pod="openshift-monitoring/metrics-server-6cfdb4569f-sxnmp" Jul 28 13:48:27.868338 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:27.867826 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/851f584e-e63a-469b-8043-194d66cce086-secret-metrics-server-tls\") pod \"metrics-server-6cfdb4569f-sxnmp\" (UID: \"851f584e-e63a-469b-8043-194d66cce086\") " pod="openshift-monitoring/metrics-server-6cfdb4569f-sxnmp" Jul 28 13:48:27.868338 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:27.867891 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/851f584e-e63a-469b-8043-194d66cce086-secret-metrics-server-client-certs\") pod \"metrics-server-6cfdb4569f-sxnmp\" (UID: \"851f584e-e63a-469b-8043-194d66cce086\") " pod="openshift-monitoring/metrics-server-6cfdb4569f-sxnmp" Jul 28 13:48:27.870517 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:27.869957 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/851f584e-e63a-469b-8043-194d66cce086-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6cfdb4569f-sxnmp\" (UID: \"851f584e-e63a-469b-8043-194d66cce086\") " pod="openshift-monitoring/metrics-server-6cfdb4569f-sxnmp" Jul 28 13:48:27.871313 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:27.871273 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/851f584e-e63a-469b-8043-194d66cce086-metrics-server-audit-profiles\") pod \"metrics-server-6cfdb4569f-sxnmp\" (UID: \"851f584e-e63a-469b-8043-194d66cce086\") " pod="openshift-monitoring/metrics-server-6cfdb4569f-sxnmp" Jul 28 13:48:27.871426 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:27.871330 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/851f584e-e63a-469b-8043-194d66cce086-audit-log\") pod \"metrics-server-6cfdb4569f-sxnmp\" (UID: \"851f584e-e63a-469b-8043-194d66cce086\") " pod="openshift-monitoring/metrics-server-6cfdb4569f-sxnmp" Jul 28 13:48:27.872689 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:27.872548 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/851f584e-e63a-469b-8043-194d66cce086-client-ca-bundle\") pod \"metrics-server-6cfdb4569f-sxnmp\" (UID: \"851f584e-e63a-469b-8043-194d66cce086\") " pod="openshift-monitoring/metrics-server-6cfdb4569f-sxnmp" Jul 28 13:48:27.872689 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:27.872665 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/851f584e-e63a-469b-8043-194d66cce086-secret-metrics-server-client-certs\") pod \"metrics-server-6cfdb4569f-sxnmp\" (UID: \"851f584e-e63a-469b-8043-194d66cce086\") " pod="openshift-monitoring/metrics-server-6cfdb4569f-sxnmp" Jul 28 13:48:27.876591 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:27.876533 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/851f584e-e63a-469b-8043-194d66cce086-secret-metrics-server-tls\") pod \"metrics-server-6cfdb4569f-sxnmp\" (UID: \"851f584e-e63a-469b-8043-194d66cce086\") " pod="openshift-monitoring/metrics-server-6cfdb4569f-sxnmp" Jul 28 13:48:27.881813 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:27.881764 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tlmj\" (UniqueName: \"kubernetes.io/projected/851f584e-e63a-469b-8043-194d66cce086-kube-api-access-8tlmj\") pod \"metrics-server-6cfdb4569f-sxnmp\" (UID: \"851f584e-e63a-469b-8043-194d66cce086\") " pod="openshift-monitoring/metrics-server-6cfdb4569f-sxnmp" Jul 28 13:48:28.084069 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.083735 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6cfdb4569f-sxnmp" Jul 28 13:48:28.190447 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.189175 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-bs6f6"] Jul 28 13:48:28.197494 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.192649 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bs6f6" Jul 28 13:48:28.197739 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.197713 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-whd6t\"" Jul 28 13:48:28.197999 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.197980 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 28 13:48:28.207655 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.207583 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-bs6f6"] Jul 28 13:48:28.251705 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.251633 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6cfdb4569f-sxnmp"] Jul 28 13:48:28.255438 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:48:28.255372 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod851f584e_e63a_469b_8043_194d66cce086.slice/crio-00397898f8c8cc9a5243444596208282ed07535dad7711326d9fa39a3b540abd WatchSource:0}: Error finding container 00397898f8c8cc9a5243444596208282ed07535dad7711326d9fa39a3b540abd: Status 404 returned error can't find the container with id 00397898f8c8cc9a5243444596208282ed07535dad7711326d9fa39a3b540abd Jul 28 13:48:28.271789 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.271756 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fc17a1fb-5683-4ce6-b54d-f164c9ca95fc-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-bs6f6\" (UID: \"fc17a1fb-5683-4ce6-b54d-f164c9ca95fc\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bs6f6" Jul 28 13:48:28.271933 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.271914 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmkmk\" (UniqueName: \"kubernetes.io/projected/fc17a1fb-5683-4ce6-b54d-f164c9ca95fc-kube-api-access-bmkmk\") pod \"monitoring-plugin-68b7fc9574-bs6f6\" (UID: \"fc17a1fb-5683-4ce6-b54d-f164c9ca95fc\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bs6f6" Jul 28 13:48:28.373564 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.372597 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fc17a1fb-5683-4ce6-b54d-f164c9ca95fc-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-bs6f6\" (UID: \"fc17a1fb-5683-4ce6-b54d-f164c9ca95fc\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bs6f6" Jul 28 13:48:28.373564 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.372768 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmkmk\" (UniqueName: \"kubernetes.io/projected/fc17a1fb-5683-4ce6-b54d-f164c9ca95fc-kube-api-access-bmkmk\") pod \"monitoring-plugin-68b7fc9574-bs6f6\" (UID: \"fc17a1fb-5683-4ce6-b54d-f164c9ca95fc\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bs6f6" Jul 28 13:48:28.377288 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.377235 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fc17a1fb-5683-4ce6-b54d-f164c9ca95fc-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-bs6f6\" (UID: \"fc17a1fb-5683-4ce6-b54d-f164c9ca95fc\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bs6f6" Jul 28 13:48:28.383428 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.383338 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmkmk\" (UniqueName: \"kubernetes.io/projected/fc17a1fb-5683-4ce6-b54d-f164c9ca95fc-kube-api-access-bmkmk\") pod \"monitoring-plugin-68b7fc9574-bs6f6\" (UID: \"fc17a1fb-5683-4ce6-b54d-f164c9ca95fc\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bs6f6" Jul 28 13:48:28.509013 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.508982 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bs6f6" Jul 28 13:48:28.663260 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.663226 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-78699f8669-pw29w"] Jul 28 13:48:28.666179 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.666150 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-bs6f6"] Jul 28 13:48:28.666331 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.666275 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-78699f8669-pw29w" Jul 28 13:48:28.669107 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.669086 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 28 13:48:28.669611 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.669346 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 28 13:48:28.669611 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.669352 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-9bb6z\"" Jul 28 13:48:28.671319 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.670750 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 28 13:48:28.672033 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.672012 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 28 13:48:28.674502 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.674472 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 28 13:48:28.681998 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.681973 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 28 13:48:28.682622 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.682603 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-78699f8669-pw29w"] Jul 28 13:48:28.768997 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.768875 2550 generic.go:356] "Generic (PLEG): container finished" podID="d43548cb-241a-490a-a94f-7c08efe068cc" containerID="d3edce344422b034828e50de178c751a65de672e87fee87b864cd0fa9a51fdc9" exitCode=0 Jul 28 13:48:28.768997 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.768953 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d43548cb-241a-490a-a94f-7c08efe068cc","Type":"ContainerDied","Data":"d3edce344422b034828e50de178c751a65de672e87fee87b864cd0fa9a51fdc9"} Jul 28 13:48:28.771597 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.771564 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbclx" event={"ID":"8daae3b7-94f7-47e0-98dd-567bde1a0038","Type":"ContainerStarted","Data":"539117add02cabf64d9712dee06aa67aa88e75b9cddda1a311c0fbf2f70e1742"} Jul 28 13:48:28.774973 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.774863 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x4rhq" event={"ID":"bc5f9125-f382-4871-92f6-fdd66afeafd3","Type":"ContainerStarted","Data":"4ca93c25e5ac04aa46ac529b9d755d34caf2e57c70786376c6b494774173abfd"} Jul 28 13:48:28.774973 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.774914 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x4rhq" event={"ID":"bc5f9125-f382-4871-92f6-fdd66afeafd3","Type":"ContainerStarted","Data":"6143c8d92f3f8e0950ca05058fc79d9b1b5981573a5ee81f48d3d3b61f483175"} Jul 28 13:48:28.776536 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.776486 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bs6f6" event={"ID":"fc17a1fb-5683-4ce6-b54d-f164c9ca95fc","Type":"ContainerStarted","Data":"4e986efb54821fc74b8c535b30717e34d99deaa684af569a8b22efa6b62e0a62"} Jul 28 13:48:28.777905 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.777878 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6cfdb4569f-sxnmp" event={"ID":"851f584e-e63a-469b-8043-194d66cce086","Type":"ContainerStarted","Data":"00397898f8c8cc9a5243444596208282ed07535dad7711326d9fa39a3b540abd"} Jul 28 13:48:28.780210 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.780174 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f4d8acff-39b5-4e12-9465-cf293804b5d6-telemeter-client-tls\") pod \"telemeter-client-78699f8669-pw29w\" (UID: \"f4d8acff-39b5-4e12-9465-cf293804b5d6\") " pod="openshift-monitoring/telemeter-client-78699f8669-pw29w" Jul 28 13:48:28.780324 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.780213 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4d8acff-39b5-4e12-9465-cf293804b5d6-serving-certs-ca-bundle\") pod \"telemeter-client-78699f8669-pw29w\" (UID: \"f4d8acff-39b5-4e12-9465-cf293804b5d6\") " pod="openshift-monitoring/telemeter-client-78699f8669-pw29w" Jul 28 13:48:28.780324 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.780258 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4d8acff-39b5-4e12-9465-cf293804b5d6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-78699f8669-pw29w\" (UID: \"f4d8acff-39b5-4e12-9465-cf293804b5d6\") " pod="openshift-monitoring/telemeter-client-78699f8669-pw29w" Jul 28 13:48:28.780440 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.780316 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f4d8acff-39b5-4e12-9465-cf293804b5d6-secret-telemeter-client\") pod \"telemeter-client-78699f8669-pw29w\" (UID: \"f4d8acff-39b5-4e12-9465-cf293804b5d6\") " pod="openshift-monitoring/telemeter-client-78699f8669-pw29w" Jul 28 13:48:28.780440 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.780366 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f4d8acff-39b5-4e12-9465-cf293804b5d6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-78699f8669-pw29w\" (UID: \"f4d8acff-39b5-4e12-9465-cf293804b5d6\") " pod="openshift-monitoring/telemeter-client-78699f8669-pw29w" Jul 28 13:48:28.780440 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.780408 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtvmt\" (UniqueName: \"kubernetes.io/projected/f4d8acff-39b5-4e12-9465-cf293804b5d6-kube-api-access-wtvmt\") pod \"telemeter-client-78699f8669-pw29w\" (UID: \"f4d8acff-39b5-4e12-9465-cf293804b5d6\") " pod="openshift-monitoring/telemeter-client-78699f8669-pw29w" Jul 28 13:48:28.780570 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.780470 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f4d8acff-39b5-4e12-9465-cf293804b5d6-federate-client-tls\") pod \"telemeter-client-78699f8669-pw29w\" (UID: \"f4d8acff-39b5-4e12-9465-cf293804b5d6\") " pod="openshift-monitoring/telemeter-client-78699f8669-pw29w" Jul 28 13:48:28.780570 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.780513 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f4d8acff-39b5-4e12-9465-cf293804b5d6-metrics-client-ca\") pod \"telemeter-client-78699f8669-pw29w\" (UID: \"f4d8acff-39b5-4e12-9465-cf293804b5d6\") " pod="openshift-monitoring/telemeter-client-78699f8669-pw29w" Jul 28 13:48:28.780965 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.780943 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-5m62f" event={"ID":"14a143a8-6917-4413-8d3b-333cb12e6f7b","Type":"ContainerStarted","Data":"f253caf31ef71995fb5b5307d60b0be9ce3e2a8bf7965d0f56a0efdccf64e531"} Jul 28 13:48:28.781061 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.780996 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-5m62f" event={"ID":"14a143a8-6917-4413-8d3b-333cb12e6f7b","Type":"ContainerStarted","Data":"9284d6e1dc9b5236ee76d2f631e400dfb3fa3e45ea9962f26ffca8ec891fdd74"} Jul 28 13:48:28.816492 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.816220 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-cbclx" podStartSLOduration=3.59403553 podStartE2EDuration="5.816200546s" totalImagesPullingTime="2.222165016s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:48:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:48:28.815307859 +0000 UTC m=+62.950504752" watchObservedRunningTime="2026-07-28 13:48:28.816200546 +0000 UTC m=+62.951397439" Jul 28 13:48:28.833773 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.833720 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-5m62f" podStartSLOduration=3.169388399 podStartE2EDuration="5.833703887s" totalImagesPullingTime="2.664315488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:48:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:48:28.833375918 +0000 UTC m=+62.968572812" watchObservedRunningTime="2026-07-28 13:48:28.833703887 +0000 UTC m=+62.968900779" Jul 28 13:48:28.853695 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.852579 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-x4rhq" podStartSLOduration=3.903187818 podStartE2EDuration="5.852561649s" totalImagesPullingTime="1.847224031s" totalInitContainerRuntime="102.1498ms" isStatefulPod=true podCreationTimestamp="2026-07-28 13:48:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:48:28.852114484 +0000 UTC m=+62.987311400" watchObservedRunningTime="2026-07-28 13:48:28.852561649 +0000 UTC m=+62.987758542" Jul 28 13:48:28.881275 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.881221 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f4d8acff-39b5-4e12-9465-cf293804b5d6-telemeter-client-tls\") pod \"telemeter-client-78699f8669-pw29w\" (UID: \"f4d8acff-39b5-4e12-9465-cf293804b5d6\") " pod="openshift-monitoring/telemeter-client-78699f8669-pw29w" Jul 28 13:48:28.881475 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.881340 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4d8acff-39b5-4e12-9465-cf293804b5d6-serving-certs-ca-bundle\") pod \"telemeter-client-78699f8669-pw29w\" (UID: \"f4d8acff-39b5-4e12-9465-cf293804b5d6\") " pod="openshift-monitoring/telemeter-client-78699f8669-pw29w" Jul 28 13:48:28.881475 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.881438 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4d8acff-39b5-4e12-9465-cf293804b5d6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-78699f8669-pw29w\" (UID: \"f4d8acff-39b5-4e12-9465-cf293804b5d6\") " pod="openshift-monitoring/telemeter-client-78699f8669-pw29w" Jul 28 13:48:28.881596 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.881534 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f4d8acff-39b5-4e12-9465-cf293804b5d6-secret-telemeter-client\") pod \"telemeter-client-78699f8669-pw29w\" (UID: \"f4d8acff-39b5-4e12-9465-cf293804b5d6\") " pod="openshift-monitoring/telemeter-client-78699f8669-pw29w" Jul 28 13:48:28.881661 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.881649 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f4d8acff-39b5-4e12-9465-cf293804b5d6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-78699f8669-pw29w\" (UID: \"f4d8acff-39b5-4e12-9465-cf293804b5d6\") " pod="openshift-monitoring/telemeter-client-78699f8669-pw29w" Jul 28 13:48:28.881731 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.881679 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtvmt\" (UniqueName: \"kubernetes.io/projected/f4d8acff-39b5-4e12-9465-cf293804b5d6-kube-api-access-wtvmt\") pod \"telemeter-client-78699f8669-pw29w\" (UID: \"f4d8acff-39b5-4e12-9465-cf293804b5d6\") " pod="openshift-monitoring/telemeter-client-78699f8669-pw29w" Jul 28 13:48:28.883165 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.881808 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f4d8acff-39b5-4e12-9465-cf293804b5d6-federate-client-tls\") pod \"telemeter-client-78699f8669-pw29w\" (UID: \"f4d8acff-39b5-4e12-9465-cf293804b5d6\") " pod="openshift-monitoring/telemeter-client-78699f8669-pw29w" Jul 28 13:48:28.883165 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.882181 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4d8acff-39b5-4e12-9465-cf293804b5d6-serving-certs-ca-bundle\") pod \"telemeter-client-78699f8669-pw29w\" (UID: \"f4d8acff-39b5-4e12-9465-cf293804b5d6\") " pod="openshift-monitoring/telemeter-client-78699f8669-pw29w" Jul 28 13:48:28.883165 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.882543 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f4d8acff-39b5-4e12-9465-cf293804b5d6-metrics-client-ca\") pod \"telemeter-client-78699f8669-pw29w\" (UID: \"f4d8acff-39b5-4e12-9465-cf293804b5d6\") " pod="openshift-monitoring/telemeter-client-78699f8669-pw29w" Jul 28 13:48:28.883433 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.883176 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4d8acff-39b5-4e12-9465-cf293804b5d6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-78699f8669-pw29w\" (UID: \"f4d8acff-39b5-4e12-9465-cf293804b5d6\") " pod="openshift-monitoring/telemeter-client-78699f8669-pw29w" Jul 28 13:48:28.883581 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.883541 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f4d8acff-39b5-4e12-9465-cf293804b5d6-metrics-client-ca\") pod \"telemeter-client-78699f8669-pw29w\" (UID: \"f4d8acff-39b5-4e12-9465-cf293804b5d6\") " pod="openshift-monitoring/telemeter-client-78699f8669-pw29w" Jul 28 13:48:28.884735 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.884713 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f4d8acff-39b5-4e12-9465-cf293804b5d6-federate-client-tls\") pod \"telemeter-client-78699f8669-pw29w\" (UID: \"f4d8acff-39b5-4e12-9465-cf293804b5d6\") " pod="openshift-monitoring/telemeter-client-78699f8669-pw29w" Jul 28 13:48:28.884831 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.884759 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f4d8acff-39b5-4e12-9465-cf293804b5d6-telemeter-client-tls\") pod \"telemeter-client-78699f8669-pw29w\" (UID: \"f4d8acff-39b5-4e12-9465-cf293804b5d6\") " pod="openshift-monitoring/telemeter-client-78699f8669-pw29w" Jul 28 13:48:28.885100 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.885076 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f4d8acff-39b5-4e12-9465-cf293804b5d6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-78699f8669-pw29w\" (UID: \"f4d8acff-39b5-4e12-9465-cf293804b5d6\") " pod="openshift-monitoring/telemeter-client-78699f8669-pw29w" Jul 28 13:48:28.885919 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.885899 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f4d8acff-39b5-4e12-9465-cf293804b5d6-secret-telemeter-client\") pod \"telemeter-client-78699f8669-pw29w\" (UID: \"f4d8acff-39b5-4e12-9465-cf293804b5d6\") " pod="openshift-monitoring/telemeter-client-78699f8669-pw29w" Jul 28 13:48:28.891221 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.891201 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtvmt\" (UniqueName: \"kubernetes.io/projected/f4d8acff-39b5-4e12-9465-cf293804b5d6-kube-api-access-wtvmt\") pod \"telemeter-client-78699f8669-pw29w\" (UID: \"f4d8acff-39b5-4e12-9465-cf293804b5d6\") " pod="openshift-monitoring/telemeter-client-78699f8669-pw29w" Jul 28 13:48:28.982924 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:28.982776 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-78699f8669-pw29w" Jul 28 13:48:29.137249 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.137199 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-78699f8669-pw29w"] Jul 28 13:48:29.140502 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:48:29.140254 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4d8acff_39b5_4e12_9465_cf293804b5d6.slice/crio-cebfc8c827816d0145471ed396e42eee7067939fe082dca32b130a07b726d018 WatchSource:0}: Error finding container cebfc8c827816d0145471ed396e42eee7067939fe082dca32b130a07b726d018: Status 404 returned error can't find the container with id cebfc8c827816d0145471ed396e42eee7067939fe082dca32b130a07b726d018 Jul 28 13:48:29.749342 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.749311 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 13:48:29.754204 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.752941 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.757662 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.757172 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 28 13:48:29.757662 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.757225 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 28 13:48:29.757662 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.757496 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 28 13:48:29.758516 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.758300 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 28 13:48:29.758624 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.758610 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 28 13:48:29.759248 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.758787 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 28 13:48:29.759248 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.758933 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 28 13:48:29.759248 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.759034 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-s2wvd\"" Jul 28 13:48:29.759248 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.759059 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-e7pgtvnm40u92\"" Jul 28 13:48:29.759248 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.759160 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 28 13:48:29.759248 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.759170 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 28 13:48:29.759792 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.759689 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 28 13:48:29.760495 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.760207 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 28 13:48:29.760879 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.760824 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 28 13:48:29.773408 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.772885 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 28 13:48:29.773408 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.773338 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 13:48:29.777287 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.777259 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 28 13:48:29.792356 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.792312 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78699f8669-pw29w" event={"ID":"f4d8acff-39b5-4e12-9465-cf293804b5d6","Type":"ContainerStarted","Data":"cebfc8c827816d0145471ed396e42eee7067939fe082dca32b130a07b726d018"} Jul 28 13:48:29.792535 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.792458 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.792594 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.792533 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.793383 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.792565 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-config\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.793551 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.793436 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.793551 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.793515 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-web-config\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.793551 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.793541 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.793717 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.793591 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hl8s8\" (UniqueName: \"kubernetes.io/projected/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-kube-api-access-hl8s8\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.793900 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.793699 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.793900 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.793743 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-config-out\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.793900 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.793768 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.793900 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.793806 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.793900 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.793828 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.793900 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.793853 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.795087 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.794201 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.795087 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.794234 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.795087 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.794526 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.795277 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.795176 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.795277 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.795215 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.795277 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.795270 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.795430 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.795295 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.896794 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.896748 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-config\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.896794 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.896791 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.897781 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.897449 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-web-config\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.897781 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.897500 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.897781 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.897563 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl8s8\" (UniqueName: \"kubernetes.io/projected/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-kube-api-access-hl8s8\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.897781 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.897663 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.897781 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.897739 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.898119 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.897940 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-config-out\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.898463 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.898439 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.898558 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.898478 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.898558 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.898514 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.898639 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.898539 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.898639 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.898599 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.898639 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.898626 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.898737 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.898685 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.898737 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.898716 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.898809 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.898731 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.898809 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.898747 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.900705 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.899086 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.900705 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.899202 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.900705 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.899304 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.900705 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.899334 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.900705 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.899364 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.900705 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.899404 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.900705 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.899941 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.903293 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.902790 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.903555 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.903501 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.904001 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.903955 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.904765 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.904557 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.905969 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.905945 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.906594 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.906573 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-web-config\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.911346 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.911298 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-config\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.911942 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.911897 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.912044 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.911970 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.912333 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.912285 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-config-out\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.912611 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.912553 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.912941 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.912894 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.913539 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.913516 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.914544 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.914465 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:29.915370 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:29.915217 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hl8s8\" (UniqueName: \"kubernetes.io/projected/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-kube-api-access-hl8s8\") pod \"prometheus-k8s-0\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:30.075491 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:30.075347 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:48:31.218354 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:31.217733 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b621445-7c29-4e1f-a55e-bc7161202598-metrics-certs\") pod \"network-metrics-daemon-khglj\" (UID: \"7b621445-7c29-4e1f-a55e-bc7161202598\") " pod="openshift-multus/network-metrics-daemon-khglj" Jul 28 13:48:31.218354 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:31.217838 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f5efacaa-1da7-4ed7-8bfa-2e805507fde7-original-pull-secret\") pod \"global-pull-secret-syncer-89dlx\" (UID: \"f5efacaa-1da7-4ed7-8bfa-2e805507fde7\") " pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:48:31.222629 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:31.222421 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 28 13:48:31.230442 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:31.224725 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 28 13:48:31.234655 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:31.234591 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f5efacaa-1da7-4ed7-8bfa-2e805507fde7-original-pull-secret\") pod \"global-pull-secret-syncer-89dlx\" (UID: \"f5efacaa-1da7-4ed7-8bfa-2e805507fde7\") " pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:48:31.237883 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:31.237836 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b621445-7c29-4e1f-a55e-bc7161202598-metrics-certs\") pod \"network-metrics-daemon-khglj\" (UID: \"7b621445-7c29-4e1f-a55e-bc7161202598\") " pod="openshift-multus/network-metrics-daemon-khglj" Jul 28 13:48:31.285806 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:31.285759 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zrb86\"" Jul 28 13:48:31.288064 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:31.287707 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dlx" Jul 28 13:48:31.289488 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:31.289337 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 13:48:31.293523 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:31.293333 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khglj" Jul 28 13:48:31.319276 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:31.318841 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzjvh\" (UniqueName: \"kubernetes.io/projected/fbc15680-5aba-470f-b2ee-fcdf596983ae-kube-api-access-fzjvh\") pod \"network-check-target-2z2sg\" (UID: \"fbc15680-5aba-470f-b2ee-fcdf596983ae\") " pod="openshift-network-diagnostics/network-check-target-2z2sg" Jul 28 13:48:31.322136 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:31.321936 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 28 13:48:31.332224 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:31.332080 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 28 13:48:31.343422 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:31.343377 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzjvh\" (UniqueName: \"kubernetes.io/projected/fbc15680-5aba-470f-b2ee-fcdf596983ae-kube-api-access-fzjvh\") pod \"network-check-target-2z2sg\" (UID: \"fbc15680-5aba-470f-b2ee-fcdf596983ae\") " pod="openshift-network-diagnostics/network-check-target-2z2sg" Jul 28 13:48:31.393839 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:31.392095 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-crbr2\"" Jul 28 13:48:31.399261 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:31.399072 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z2sg" Jul 28 13:48:31.807609 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:31.807565 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d43548cb-241a-490a-a94f-7c08efe068cc","Type":"ContainerStarted","Data":"19e39320428fea534d2d3d31970c3ca4143d49e5408aa7154cc367b34daee637"} Jul 28 13:48:31.807809 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:31.807679 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d43548cb-241a-490a-a94f-7c08efe068cc","Type":"ContainerStarted","Data":"4bfe0b879d58e16a27da54b68097023a039c1caf8e893646119e34553f799fe1"} Jul 28 13:48:31.809957 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:31.809861 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-ff6496dd8-k76hx" event={"ID":"4b4c841c-be47-457a-96d4-c2bef7851671","Type":"ContainerStarted","Data":"97098a9a18f21a9256d4a2586b7e921fb620a011261969c0b5cfc109da5254b4"} Jul 28 13:48:31.809957 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:31.809922 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-ff6496dd8-k76hx" event={"ID":"4b4c841c-be47-457a-96d4-c2bef7851671","Type":"ContainerStarted","Data":"c6770ce13266da624f821595dc284cc25d49840b3a68303c450618f7b9e728f3"} Jul 28 13:48:31.811422 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:31.811385 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d","Type":"ContainerStarted","Data":"d992fc72e69f52c7315350e8f29b8ec7012a92d265c5af66be4d6fc76389f4a9"} Jul 28 13:48:32.383464 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:32.383422 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-khglj"] Jul 28 13:48:32.792484 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:48:32.792225 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b621445_7c29_4e1f_a55e_bc7161202598.slice/crio-377635560c7ff50eabdf187047517adc6c228eec767d3547ac0afb884588cef8 WatchSource:0}: Error finding container 377635560c7ff50eabdf187047517adc6c228eec767d3547ac0afb884588cef8: Status 404 returned error can't find the container with id 377635560c7ff50eabdf187047517adc6c228eec767d3547ac0afb884588cef8 Jul 28 13:48:32.821233 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:32.821159 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-khglj" event={"ID":"7b621445-7c29-4e1f-a55e-bc7161202598","Type":"ContainerStarted","Data":"377635560c7ff50eabdf187047517adc6c228eec767d3547ac0afb884588cef8"} Jul 28 13:48:32.983197 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:32.983124 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2z2sg"] Jul 28 13:48:32.986801 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:48:32.986532 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbc15680_5aba_470f_b2ee_fcdf596983ae.slice/crio-04e6338259678aa47952981aa1371a81d2134a4fb43ac7f47a1c1f2a04e92517 WatchSource:0}: Error finding container 04e6338259678aa47952981aa1371a81d2134a4fb43ac7f47a1c1f2a04e92517: Status 404 returned error can't find the container with id 04e6338259678aa47952981aa1371a81d2134a4fb43ac7f47a1c1f2a04e92517 Jul 28 13:48:32.997357 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:32.997330 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-89dlx"] Jul 28 13:48:33.005120 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:48:33.004905 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5efacaa_1da7_4ed7_8bfa_2e805507fde7.slice/crio-cc05e3eee28947868ba408f9a5ea14d0f40809a9b2513184b0f1315ef59e76e7 WatchSource:0}: Error finding container cc05e3eee28947868ba408f9a5ea14d0f40809a9b2513184b0f1315ef59e76e7: Status 404 returned error can't find the container with id cc05e3eee28947868ba408f9a5ea14d0f40809a9b2513184b0f1315ef59e76e7 Jul 28 13:48:33.833365 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:33.833325 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d43548cb-241a-490a-a94f-7c08efe068cc","Type":"ContainerStarted","Data":"d2f997e70df7b051cf373acb4f9509a9fbcdeb79e00706d853bf4ac143f723dd"} Jul 28 13:48:33.833846 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:33.833477 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d43548cb-241a-490a-a94f-7c08efe068cc","Type":"ContainerStarted","Data":"b17f6644d14101b3fd843c5b062abf07c2be9074aba09899390e2fa69080d02b"} Jul 28 13:48:33.833846 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:33.833529 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d43548cb-241a-490a-a94f-7c08efe068cc","Type":"ContainerStarted","Data":"e02897857e885176cff1ccc7dcea3a1f61f3b608db349355ee9eec4d51110a28"} Jul 28 13:48:33.838707 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:33.838672 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-ff6496dd8-k76hx" event={"ID":"4b4c841c-be47-457a-96d4-c2bef7851671","Type":"ContainerStarted","Data":"aa3b7a4e9225aaf95c4cd59627f9f955a79645fb9ff0d134b3e61c9bfd19e671"} Jul 28 13:48:33.842289 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:33.841005 2550 generic.go:356] "Generic (PLEG): container finished" podID="cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" containerID="727c52f4873cc6b2b374041bd49ccdb19cf16ea52fbacd85505e857623c03963" exitCode=0 Jul 28 13:48:33.842289 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:33.841199 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d","Type":"ContainerDied","Data":"727c52f4873cc6b2b374041bd49ccdb19cf16ea52fbacd85505e857623c03963"} Jul 28 13:48:33.845731 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:33.845671 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-89dlx" event={"ID":"f5efacaa-1da7-4ed7-8bfa-2e805507fde7","Type":"ContainerStarted","Data":"cc05e3eee28947868ba408f9a5ea14d0f40809a9b2513184b0f1315ef59e76e7"} Jul 28 13:48:33.849465 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:33.849416 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2z2sg" event={"ID":"fbc15680-5aba-470f-b2ee-fcdf596983ae","Type":"ContainerStarted","Data":"04e6338259678aa47952981aa1371a81d2134a4fb43ac7f47a1c1f2a04e92517"} Jul 28 13:48:33.851591 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:33.851566 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bs6f6" event={"ID":"fc17a1fb-5683-4ce6-b54d-f164c9ca95fc","Type":"ContainerStarted","Data":"1a409d851cb926d1c2bf2d7b74466369ef27eeff6672d4e9c939ace4c8153853"} Jul 28 13:48:33.852041 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:33.851976 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bs6f6" Jul 28 13:48:33.855100 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:33.855073 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6cfdb4569f-sxnmp" event={"ID":"851f584e-e63a-469b-8043-194d66cce086","Type":"ContainerStarted","Data":"812ae597dd0862a13744a6327a64aada20cb1a672e0c16102716a3181fce23c7"} Jul 28 13:48:33.859513 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:33.859453 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bs6f6" Jul 28 13:48:33.860861 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:33.860838 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78699f8669-pw29w" event={"ID":"f4d8acff-39b5-4e12-9465-cf293804b5d6","Type":"ContainerStarted","Data":"e22dddf0042254ab63cdbe01a0ebc8fa44714098fc1250590ac76c048915d4b7"} Jul 28 13:48:33.860957 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:33.860891 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78699f8669-pw29w" event={"ID":"f4d8acff-39b5-4e12-9465-cf293804b5d6","Type":"ContainerStarted","Data":"87af1a2c62a11348dfef6fcb41ec2f2c53e6d0def88ca73ffb3ee84f64654678"} Jul 28 13:48:33.860957 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:33.860921 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78699f8669-pw29w" event={"ID":"f4d8acff-39b5-4e12-9465-cf293804b5d6","Type":"ContainerStarted","Data":"696bc5fc37318f32714d6046c25e15f75b72f0818f660f0e4cd65309e31c05a1"} Jul 28 13:48:33.905323 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:33.903518 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6cfdb4569f-sxnmp" podStartSLOduration=2.349141743 podStartE2EDuration="6.903498047s" totalImagesPullingTime="4.554356304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:48:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:48:33.90311622 +0000 UTC m=+68.038313113" watchObservedRunningTime="2026-07-28 13:48:33.903498047 +0000 UTC m=+68.038694940" Jul 28 13:48:33.921336 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:33.921000 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bs6f6" podStartSLOduration=1.789472127 podStartE2EDuration="5.920977346s" totalImagesPullingTime="4.131505219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:48:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:48:33.918860916 +0000 UTC m=+68.054058019" watchObservedRunningTime="2026-07-28 13:48:33.920977346 +0000 UTC m=+68.056174238" Jul 28 13:48:33.945656 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:33.944754 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-78699f8669-pw29w" podStartSLOduration=2.20363743 podStartE2EDuration="5.944728891s" totalImagesPullingTime="3.741091461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:48:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:48:33.942386141 +0000 UTC m=+68.077583033" watchObservedRunningTime="2026-07-28 13:48:33.944728891 +0000 UTC m=+68.079925766" Jul 28 13:48:34.887094 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:34.884671 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d43548cb-241a-490a-a94f-7c08efe068cc","Type":"ContainerStarted","Data":"c1e9a064e55b8786635e4087071da82b1c70de20cd1324ce6b43a4552897a8c6"} Jul 28 13:48:34.916369 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:34.916305 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.862142938 podStartE2EDuration="10.916281996s" totalImagesPullingTime="6.021334795s" totalInitContainerRuntime="32.804263ms" isStatefulPod=false podCreationTimestamp="2026-07-28 13:48:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:48:34.915106204 +0000 UTC m=+69.050303098" watchObservedRunningTime="2026-07-28 13:48:34.916281996 +0000 UTC m=+69.051478891" Jul 28 13:48:35.903839 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:35.903799 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-ff6496dd8-k76hx" event={"ID":"4b4c841c-be47-457a-96d4-c2bef7851671","Type":"ContainerStarted","Data":"2dd405dc05ce6ce193e03ccf10553ee01b897df32547ee26a3c30344ad584845"} Jul 28 13:48:35.904276 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:35.903894 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-ff6496dd8-k76hx" event={"ID":"4b4c841c-be47-457a-96d4-c2bef7851671","Type":"ContainerStarted","Data":"4b3be23633e4ffe5759ef3c2054164ebedea52f51e2d97a7c5088750537c2dde"} Jul 28 13:48:35.904276 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:35.903976 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-ff6496dd8-k76hx" event={"ID":"4b4c841c-be47-457a-96d4-c2bef7851671","Type":"ContainerStarted","Data":"5fa2072dfb3d39fe8ede939f02200d881c9ae98b5859b737a52fb5f298352361"} Jul 28 13:48:35.905417 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:35.905004 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-ff6496dd8-k76hx" Jul 28 13:48:35.933107 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:35.931714 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-ff6496dd8-k76hx" podStartSLOduration=5.380471361 podStartE2EDuration="10.931690078s" totalImagesPullingTime="5.551218717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:48:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:48:35.928288452 +0000 UTC m=+70.063485347" watchObservedRunningTime="2026-07-28 13:48:35.931690078 +0000 UTC m=+70.066886972" Jul 28 13:48:36.920029 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:36.919994 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-ff6496dd8-k76hx" Jul 28 13:48:48.084487 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:48.084373 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6cfdb4569f-sxnmp" Jul 28 13:48:48.084487 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:48.084443 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6cfdb4569f-sxnmp" Jul 28 13:48:48.973806 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:48.973378 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-9ws9v" event={"ID":"30347318-34fb-44a6-a955-9b3774e1f73c","Type":"ContainerStarted","Data":"71ef2c60775ffd227c05a09d5f05a27825bc269b08870d16cc80cc8564fb711b"} Jul 28 13:48:48.973806 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:48.973600 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-9ws9v" Jul 28 13:48:48.975609 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:48.975576 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-khglj" event={"ID":"7b621445-7c29-4e1f-a55e-bc7161202598","Type":"ContainerStarted","Data":"caac383d0428928f1d64b7e0d6b5741a34fb45e2403c2a34d1f67945d865943e"} Jul 28 13:48:48.975746 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:48.975628 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-khglj" event={"ID":"7b621445-7c29-4e1f-a55e-bc7161202598","Type":"ContainerStarted","Data":"65e101df8f44e83d5475378c15414fa38ba4b16c6fb99bc842fb6e93344b611b"} Jul 28 13:48:48.978862 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:48.978837 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d","Type":"ContainerStarted","Data":"0b3336c901521ece0f2cc46d92ebe011025853bf00ca345f370523466da91622"} Jul 28 13:48:48.978975 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:48.978908 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d","Type":"ContainerStarted","Data":"672b9bb96f6aabbab28f7255fb823be808212cf477650976f77ba47d7dc55fdf"} Jul 28 13:48:48.978975 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:48.978957 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d","Type":"ContainerStarted","Data":"091186c4a45bd814266ac30e567725721c206e48735026863abe79f7742e2101"} Jul 28 13:48:48.979097 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:48.978997 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d","Type":"ContainerStarted","Data":"b0f3145a81cc40b101d3797e3ccb2968894ee11d801f5d32fc338d47d802f1fc"} Jul 28 13:48:48.981109 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:48.981081 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-89dlx" event={"ID":"f5efacaa-1da7-4ed7-8bfa-2e805507fde7","Type":"ContainerStarted","Data":"0e715524b088bb5567a2654fb6477113a5df60e67d7f0dbc16146d74cd4855b3"} Jul 28 13:48:48.982440 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:48.982371 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-9ws9v" Jul 28 13:48:48.982833 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:48.982807 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2z2sg" event={"ID":"fbc15680-5aba-470f-b2ee-fcdf596983ae","Type":"ContainerStarted","Data":"fcf10d586174f78dc39fef47a197aed5e25f122acbc85b59ab212597ce47984c"} Jul 28 13:48:48.982908 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:48.982860 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2z2sg" Jul 28 13:48:48.997073 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:48.997015 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-9ws9v" podStartSLOduration=2.893361987 podStartE2EDuration="23.996993573s" totalImagesPullingTime="21.103631586s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:48:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:48:48.996919989 +0000 UTC m=+83.132116880" watchObservedRunningTime="2026-07-28 13:48:48.996993573 +0000 UTC m=+83.132190466" Jul 28 13:48:49.037447 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:49.037375 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2z2sg" podStartSLOduration=67.676161548 podStartE2EDuration="1m23.037353539s" totalImagesPullingTime="15.361191991s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:47:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:48:49.016358481 +0000 UTC m=+83.151555373" watchObservedRunningTime="2026-07-28 13:48:49.037353539 +0000 UTC m=+83.172550433" Jul 28 13:48:49.055471 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:49.055306 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-89dlx" podStartSLOduration=67.66002717 podStartE2EDuration="1m23.055285475s" totalImagesPullingTime="15.395258305s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 13:47:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:48:49.053157644 +0000 UTC m=+83.188354539" watchObservedRunningTime="2026-07-28 13:48:49.055285475 +0000 UTC m=+83.190482368" Jul 28 13:48:49.075761 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:49.075196 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-khglj" podStartSLOduration=67.519823875 podStartE2EDuration="1m23.075172486s" totalImagesPullingTime="15.555348611s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:47:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:48:49.073339886 +0000 UTC m=+83.208536778" watchObservedRunningTime="2026-07-28 13:48:49.075172486 +0000 UTC m=+83.210369380" Jul 28 13:48:49.993385 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:49.993343 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d","Type":"ContainerStarted","Data":"4bda1c363205d9aa2c8da95e0bebd585c97333b0529628384da036e5bc789d6a"} Jul 28 13:48:49.994015 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:49.993442 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d","Type":"ContainerStarted","Data":"ba84ae31980f080e56ec77a7865406afa2bfe3175cf19a429e998c68acfa325d"} Jul 28 13:48:50.029166 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:50.029101 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=6.467837695 podStartE2EDuration="21.029080709s" totalImagesPullingTime="14.530892549s" totalInitContainerRuntime="30.350465ms" isStatefulPod=false podCreationTimestamp="2026-07-28 13:48:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:48:50.026308829 +0000 UTC m=+84.161505725" watchObservedRunningTime="2026-07-28 13:48:50.029080709 +0000 UTC m=+84.164277604" Jul 28 13:48:50.076210 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:48:50.076170 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:08.089770 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:08.089734 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6cfdb4569f-sxnmp" Jul 28 13:49:08.093494 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:08.093475 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6cfdb4569f-sxnmp" Jul 28 13:49:19.996051 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:19.996018 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2z2sg" Jul 28 13:49:30.075785 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:30.075746 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:30.132531 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:30.132504 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:30.189909 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:30.189883 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:43.889147 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:43.885821 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 13:49:43.889147 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:43.886364 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d43548cb-241a-490a-a94f-7c08efe068cc" containerName="alertmanager" containerID="cri-o://4bfe0b879d58e16a27da54b68097023a039c1caf8e893646119e34553f799fe1" gracePeriod=120 Jul 28 13:49:43.889147 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:43.886707 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d43548cb-241a-490a-a94f-7c08efe068cc" containerName="kube-rbac-proxy" containerID="cri-o://b17f6644d14101b3fd843c5b062abf07c2be9074aba09899390e2fa69080d02b" gracePeriod=120 Jul 28 13:49:43.889147 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:43.886871 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d43548cb-241a-490a-a94f-7c08efe068cc" containerName="prom-label-proxy" containerID="cri-o://c1e9a064e55b8786635e4087071da82b1c70de20cd1324ce6b43a4552897a8c6" gracePeriod=120 Jul 28 13:49:43.889147 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:43.886910 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d43548cb-241a-490a-a94f-7c08efe068cc" containerName="kube-rbac-proxy-web" containerID="cri-o://e02897857e885176cff1ccc7dcea3a1f61f3b608db349355ee9eec4d51110a28" gracePeriod=120 Jul 28 13:49:43.889147 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:43.886975 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d43548cb-241a-490a-a94f-7c08efe068cc" containerName="kube-rbac-proxy-metric" containerID="cri-o://d2f997e70df7b051cf373acb4f9509a9fbcdeb79e00706d853bf4ac143f723dd" gracePeriod=120 Jul 28 13:49:43.889147 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:43.887042 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d43548cb-241a-490a-a94f-7c08efe068cc" containerName="config-reloader" containerID="cri-o://19e39320428fea534d2d3d31970c3ca4143d49e5408aa7154cc367b34daee637" gracePeriod=120 Jul 28 13:49:44.237560 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:44.237524 2550 generic.go:356] "Generic (PLEG): container finished" podID="d43548cb-241a-490a-a94f-7c08efe068cc" containerID="c1e9a064e55b8786635e4087071da82b1c70de20cd1324ce6b43a4552897a8c6" exitCode=0 Jul 28 13:49:44.237560 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:44.237554 2550 generic.go:356] "Generic (PLEG): container finished" podID="d43548cb-241a-490a-a94f-7c08efe068cc" containerID="b17f6644d14101b3fd843c5b062abf07c2be9074aba09899390e2fa69080d02b" exitCode=0 Jul 28 13:49:44.237560 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:44.237565 2550 generic.go:356] "Generic (PLEG): container finished" podID="d43548cb-241a-490a-a94f-7c08efe068cc" containerID="19e39320428fea534d2d3d31970c3ca4143d49e5408aa7154cc367b34daee637" exitCode=0 Jul 28 13:49:44.237780 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:44.237574 2550 generic.go:356] "Generic (PLEG): container finished" podID="d43548cb-241a-490a-a94f-7c08efe068cc" containerID="4bfe0b879d58e16a27da54b68097023a039c1caf8e893646119e34553f799fe1" exitCode=0 Jul 28 13:49:44.237780 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:44.237577 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d43548cb-241a-490a-a94f-7c08efe068cc","Type":"ContainerDied","Data":"c1e9a064e55b8786635e4087071da82b1c70de20cd1324ce6b43a4552897a8c6"} Jul 28 13:49:44.237780 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:44.237645 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d43548cb-241a-490a-a94f-7c08efe068cc","Type":"ContainerDied","Data":"b17f6644d14101b3fd843c5b062abf07c2be9074aba09899390e2fa69080d02b"} Jul 28 13:49:44.237780 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:44.237672 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d43548cb-241a-490a-a94f-7c08efe068cc","Type":"ContainerDied","Data":"19e39320428fea534d2d3d31970c3ca4143d49e5408aa7154cc367b34daee637"} Jul 28 13:49:44.237780 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:44.237698 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d43548cb-241a-490a-a94f-7c08efe068cc","Type":"ContainerDied","Data":"4bfe0b879d58e16a27da54b68097023a039c1caf8e893646119e34553f799fe1"} Jul 28 13:49:45.139545 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.139517 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.208187 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.208108 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-cluster-tls-config\") pod \"d43548cb-241a-490a-a94f-7c08efe068cc\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " Jul 28 13:49:45.208187 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.208144 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-secret-alertmanager-main-tls\") pod \"d43548cb-241a-490a-a94f-7c08efe068cc\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " Jul 28 13:49:45.208187 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.208178 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d43548cb-241a-490a-a94f-7c08efe068cc-tls-assets\") pod \"d43548cb-241a-490a-a94f-7c08efe068cc\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " Jul 28 13:49:45.208483 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.208206 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"d43548cb-241a-490a-a94f-7c08efe068cc\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " Jul 28 13:49:45.208483 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.208454 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-web-config\") pod \"d43548cb-241a-490a-a94f-7c08efe068cc\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " Jul 28 13:49:45.208593 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.208524 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mn8nb\" (UniqueName: \"kubernetes.io/projected/d43548cb-241a-490a-a94f-7c08efe068cc-kube-api-access-mn8nb\") pod \"d43548cb-241a-490a-a94f-7c08efe068cc\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " Jul 28 13:49:45.208593 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.208562 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d43548cb-241a-490a-a94f-7c08efe068cc-metrics-client-ca\") pod \"d43548cb-241a-490a-a94f-7c08efe068cc\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " Jul 28 13:49:45.208697 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.208602 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d43548cb-241a-490a-a94f-7c08efe068cc-config-out\") pod \"d43548cb-241a-490a-a94f-7c08efe068cc\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " Jul 28 13:49:45.208697 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.208662 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d43548cb-241a-490a-a94f-7c08efe068cc-alertmanager-trusted-ca-bundle\") pod \"d43548cb-241a-490a-a94f-7c08efe068cc\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " Jul 28 13:49:45.208807 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.208707 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-secret-alertmanager-kube-rbac-proxy-web\") pod \"d43548cb-241a-490a-a94f-7c08efe068cc\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " Jul 28 13:49:45.208807 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.208735 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d43548cb-241a-490a-a94f-7c08efe068cc-alertmanager-main-db\") pod \"d43548cb-241a-490a-a94f-7c08efe068cc\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " Jul 28 13:49:45.208807 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.208759 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-secret-alertmanager-kube-rbac-proxy\") pod \"d43548cb-241a-490a-a94f-7c08efe068cc\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " Jul 28 13:49:45.208807 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.208805 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-config-volume\") pod \"d43548cb-241a-490a-a94f-7c08efe068cc\" (UID: \"d43548cb-241a-490a-a94f-7c08efe068cc\") " Jul 28 13:49:45.209569 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.209244 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d43548cb-241a-490a-a94f-7c08efe068cc-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "d43548cb-241a-490a-a94f-7c08efe068cc" (UID: "d43548cb-241a-490a-a94f-7c08efe068cc"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 13:49:45.209913 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.209877 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d43548cb-241a-490a-a94f-7c08efe068cc-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "d43548cb-241a-490a-a94f-7c08efe068cc" (UID: "d43548cb-241a-490a-a94f-7c08efe068cc"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 13:49:45.210051 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.210021 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d43548cb-241a-490a-a94f-7c08efe068cc-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "d43548cb-241a-490a-a94f-7c08efe068cc" (UID: "d43548cb-241a-490a-a94f-7c08efe068cc"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 13:49:45.212599 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.212554 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "d43548cb-241a-490a-a94f-7c08efe068cc" (UID: "d43548cb-241a-490a-a94f-7c08efe068cc"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 13:49:45.212599 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.212588 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d43548cb-241a-490a-a94f-7c08efe068cc-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "d43548cb-241a-490a-a94f-7c08efe068cc" (UID: "d43548cb-241a-490a-a94f-7c08efe068cc"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 13:49:45.212766 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.212745 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d43548cb-241a-490a-a94f-7c08efe068cc-kube-api-access-mn8nb" (OuterVolumeSpecName: "kube-api-access-mn8nb") pod "d43548cb-241a-490a-a94f-7c08efe068cc" (UID: "d43548cb-241a-490a-a94f-7c08efe068cc"). InnerVolumeSpecName "kube-api-access-mn8nb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 13:49:45.212826 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.212784 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-config-volume" (OuterVolumeSpecName: "config-volume") pod "d43548cb-241a-490a-a94f-7c08efe068cc" (UID: "d43548cb-241a-490a-a94f-7c08efe068cc"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 13:49:45.213268 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.213227 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "d43548cb-241a-490a-a94f-7c08efe068cc" (UID: "d43548cb-241a-490a-a94f-7c08efe068cc"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 13:49:45.213358 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.213337 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "d43548cb-241a-490a-a94f-7c08efe068cc" (UID: "d43548cb-241a-490a-a94f-7c08efe068cc"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 13:49:45.213540 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.213518 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "d43548cb-241a-490a-a94f-7c08efe068cc" (UID: "d43548cb-241a-490a-a94f-7c08efe068cc"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 13:49:45.213799 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.213779 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d43548cb-241a-490a-a94f-7c08efe068cc-config-out" (OuterVolumeSpecName: "config-out") pod "d43548cb-241a-490a-a94f-7c08efe068cc" (UID: "d43548cb-241a-490a-a94f-7c08efe068cc"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 13:49:45.216963 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.216933 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "d43548cb-241a-490a-a94f-7c08efe068cc" (UID: "d43548cb-241a-490a-a94f-7c08efe068cc"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 13:49:45.224587 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.224559 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-web-config" (OuterVolumeSpecName: "web-config") pod "d43548cb-241a-490a-a94f-7c08efe068cc" (UID: "d43548cb-241a-490a-a94f-7c08efe068cc"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 13:49:45.245010 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.244985 2550 generic.go:356] "Generic (PLEG): container finished" podID="d43548cb-241a-490a-a94f-7c08efe068cc" containerID="d2f997e70df7b051cf373acb4f9509a9fbcdeb79e00706d853bf4ac143f723dd" exitCode=0 Jul 28 13:49:45.245010 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.245007 2550 generic.go:356] "Generic (PLEG): container finished" podID="d43548cb-241a-490a-a94f-7c08efe068cc" containerID="e02897857e885176cff1ccc7dcea3a1f61f3b608db349355ee9eec4d51110a28" exitCode=0 Jul 28 13:49:45.245163 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.245058 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.245163 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.245064 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d43548cb-241a-490a-a94f-7c08efe068cc","Type":"ContainerDied","Data":"d2f997e70df7b051cf373acb4f9509a9fbcdeb79e00706d853bf4ac143f723dd"} Jul 28 13:49:45.245163 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.245136 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d43548cb-241a-490a-a94f-7c08efe068cc","Type":"ContainerDied","Data":"e02897857e885176cff1ccc7dcea3a1f61f3b608db349355ee9eec4d51110a28"} Jul 28 13:49:45.245163 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.245163 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d43548cb-241a-490a-a94f-7c08efe068cc","Type":"ContainerDied","Data":"6d41498fb5f3f490a64bb54ddbeb0e3c1f59bcfa21b297253c3ae8b06bf40dfc"} Jul 28 13:49:45.245302 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.245205 2550 scope.go:117] "RemoveContainer" containerID="c1e9a064e55b8786635e4087071da82b1c70de20cd1324ce6b43a4552897a8c6" Jul 28 13:49:45.252819 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.252807 2550 scope.go:117] "RemoveContainer" containerID="d2f997e70df7b051cf373acb4f9509a9fbcdeb79e00706d853bf4ac143f723dd" Jul 28 13:49:45.259823 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.259808 2550 scope.go:117] "RemoveContainer" containerID="b17f6644d14101b3fd843c5b062abf07c2be9074aba09899390e2fa69080d02b" Jul 28 13:49:45.266500 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.266482 2550 scope.go:117] "RemoveContainer" containerID="e02897857e885176cff1ccc7dcea3a1f61f3b608db349355ee9eec4d51110a28" Jul 28 13:49:45.272446 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.272424 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 13:49:45.274189 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.274164 2550 scope.go:117] "RemoveContainer" containerID="19e39320428fea534d2d3d31970c3ca4143d49e5408aa7154cc367b34daee637" Jul 28 13:49:45.277370 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.277348 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 13:49:45.281834 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.281817 2550 scope.go:117] "RemoveContainer" containerID="4bfe0b879d58e16a27da54b68097023a039c1caf8e893646119e34553f799fe1" Jul 28 13:49:45.288844 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.288824 2550 scope.go:117] "RemoveContainer" containerID="d3edce344422b034828e50de178c751a65de672e87fee87b864cd0fa9a51fdc9" Jul 28 13:49:45.295475 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.295458 2550 scope.go:117] "RemoveContainer" containerID="c1e9a064e55b8786635e4087071da82b1c70de20cd1324ce6b43a4552897a8c6" Jul 28 13:49:45.295727 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:49:45.295707 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1e9a064e55b8786635e4087071da82b1c70de20cd1324ce6b43a4552897a8c6\": container with ID starting with c1e9a064e55b8786635e4087071da82b1c70de20cd1324ce6b43a4552897a8c6 not found: ID does not exist" containerID="c1e9a064e55b8786635e4087071da82b1c70de20cd1324ce6b43a4552897a8c6" Jul 28 13:49:45.295798 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.295740 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1e9a064e55b8786635e4087071da82b1c70de20cd1324ce6b43a4552897a8c6"} err="failed to get container status \"c1e9a064e55b8786635e4087071da82b1c70de20cd1324ce6b43a4552897a8c6\": rpc error: code = NotFound desc = could not find container \"c1e9a064e55b8786635e4087071da82b1c70de20cd1324ce6b43a4552897a8c6\": container with ID starting with c1e9a064e55b8786635e4087071da82b1c70de20cd1324ce6b43a4552897a8c6 not found: ID does not exist" Jul 28 13:49:45.295798 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.295785 2550 scope.go:117] "RemoveContainer" containerID="d2f997e70df7b051cf373acb4f9509a9fbcdeb79e00706d853bf4ac143f723dd" Jul 28 13:49:45.296047 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:49:45.296029 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2f997e70df7b051cf373acb4f9509a9fbcdeb79e00706d853bf4ac143f723dd\": container with ID starting with d2f997e70df7b051cf373acb4f9509a9fbcdeb79e00706d853bf4ac143f723dd not found: ID does not exist" containerID="d2f997e70df7b051cf373acb4f9509a9fbcdeb79e00706d853bf4ac143f723dd" Jul 28 13:49:45.296085 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.296054 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2f997e70df7b051cf373acb4f9509a9fbcdeb79e00706d853bf4ac143f723dd"} err="failed to get container status \"d2f997e70df7b051cf373acb4f9509a9fbcdeb79e00706d853bf4ac143f723dd\": rpc error: code = NotFound desc = could not find container \"d2f997e70df7b051cf373acb4f9509a9fbcdeb79e00706d853bf4ac143f723dd\": container with ID starting with d2f997e70df7b051cf373acb4f9509a9fbcdeb79e00706d853bf4ac143f723dd not found: ID does not exist" Jul 28 13:49:45.296085 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.296071 2550 scope.go:117] "RemoveContainer" containerID="b17f6644d14101b3fd843c5b062abf07c2be9074aba09899390e2fa69080d02b" Jul 28 13:49:45.296269 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:49:45.296252 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b17f6644d14101b3fd843c5b062abf07c2be9074aba09899390e2fa69080d02b\": container with ID starting with b17f6644d14101b3fd843c5b062abf07c2be9074aba09899390e2fa69080d02b not found: ID does not exist" containerID="b17f6644d14101b3fd843c5b062abf07c2be9074aba09899390e2fa69080d02b" Jul 28 13:49:45.296335 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.296277 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b17f6644d14101b3fd843c5b062abf07c2be9074aba09899390e2fa69080d02b"} err="failed to get container status \"b17f6644d14101b3fd843c5b062abf07c2be9074aba09899390e2fa69080d02b\": rpc error: code = NotFound desc = could not find container \"b17f6644d14101b3fd843c5b062abf07c2be9074aba09899390e2fa69080d02b\": container with ID starting with b17f6644d14101b3fd843c5b062abf07c2be9074aba09899390e2fa69080d02b not found: ID does not exist" Jul 28 13:49:45.296335 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.296299 2550 scope.go:117] "RemoveContainer" containerID="e02897857e885176cff1ccc7dcea3a1f61f3b608db349355ee9eec4d51110a28" Jul 28 13:49:45.296554 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:49:45.296538 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e02897857e885176cff1ccc7dcea3a1f61f3b608db349355ee9eec4d51110a28\": container with ID starting with e02897857e885176cff1ccc7dcea3a1f61f3b608db349355ee9eec4d51110a28 not found: ID does not exist" containerID="e02897857e885176cff1ccc7dcea3a1f61f3b608db349355ee9eec4d51110a28" Jul 28 13:49:45.296601 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.296558 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e02897857e885176cff1ccc7dcea3a1f61f3b608db349355ee9eec4d51110a28"} err="failed to get container status \"e02897857e885176cff1ccc7dcea3a1f61f3b608db349355ee9eec4d51110a28\": rpc error: code = NotFound desc = could not find container \"e02897857e885176cff1ccc7dcea3a1f61f3b608db349355ee9eec4d51110a28\": container with ID starting with e02897857e885176cff1ccc7dcea3a1f61f3b608db349355ee9eec4d51110a28 not found: ID does not exist" Jul 28 13:49:45.296601 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.296571 2550 scope.go:117] "RemoveContainer" containerID="19e39320428fea534d2d3d31970c3ca4143d49e5408aa7154cc367b34daee637" Jul 28 13:49:45.296810 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:49:45.296794 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19e39320428fea534d2d3d31970c3ca4143d49e5408aa7154cc367b34daee637\": container with ID starting with 19e39320428fea534d2d3d31970c3ca4143d49e5408aa7154cc367b34daee637 not found: ID does not exist" containerID="19e39320428fea534d2d3d31970c3ca4143d49e5408aa7154cc367b34daee637" Jul 28 13:49:45.296854 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.296814 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19e39320428fea534d2d3d31970c3ca4143d49e5408aa7154cc367b34daee637"} err="failed to get container status \"19e39320428fea534d2d3d31970c3ca4143d49e5408aa7154cc367b34daee637\": rpc error: code = NotFound desc = could not find container \"19e39320428fea534d2d3d31970c3ca4143d49e5408aa7154cc367b34daee637\": container with ID starting with 19e39320428fea534d2d3d31970c3ca4143d49e5408aa7154cc367b34daee637 not found: ID does not exist" Jul 28 13:49:45.296854 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.296829 2550 scope.go:117] "RemoveContainer" containerID="4bfe0b879d58e16a27da54b68097023a039c1caf8e893646119e34553f799fe1" Jul 28 13:49:45.297080 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:49:45.297062 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bfe0b879d58e16a27da54b68097023a039c1caf8e893646119e34553f799fe1\": container with ID starting with 4bfe0b879d58e16a27da54b68097023a039c1caf8e893646119e34553f799fe1 not found: ID does not exist" containerID="4bfe0b879d58e16a27da54b68097023a039c1caf8e893646119e34553f799fe1" Jul 28 13:49:45.297139 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.297086 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bfe0b879d58e16a27da54b68097023a039c1caf8e893646119e34553f799fe1"} err="failed to get container status \"4bfe0b879d58e16a27da54b68097023a039c1caf8e893646119e34553f799fe1\": rpc error: code = NotFound desc = could not find container \"4bfe0b879d58e16a27da54b68097023a039c1caf8e893646119e34553f799fe1\": container with ID starting with 4bfe0b879d58e16a27da54b68097023a039c1caf8e893646119e34553f799fe1 not found: ID does not exist" Jul 28 13:49:45.297139 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.297106 2550 scope.go:117] "RemoveContainer" containerID="d3edce344422b034828e50de178c751a65de672e87fee87b864cd0fa9a51fdc9" Jul 28 13:49:45.297318 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:49:45.297303 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3edce344422b034828e50de178c751a65de672e87fee87b864cd0fa9a51fdc9\": container with ID starting with d3edce344422b034828e50de178c751a65de672e87fee87b864cd0fa9a51fdc9 not found: ID does not exist" containerID="d3edce344422b034828e50de178c751a65de672e87fee87b864cd0fa9a51fdc9" Jul 28 13:49:45.297354 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.297321 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3edce344422b034828e50de178c751a65de672e87fee87b864cd0fa9a51fdc9"} err="failed to get container status \"d3edce344422b034828e50de178c751a65de672e87fee87b864cd0fa9a51fdc9\": rpc error: code = NotFound desc = could not find container \"d3edce344422b034828e50de178c751a65de672e87fee87b864cd0fa9a51fdc9\": container with ID starting with d3edce344422b034828e50de178c751a65de672e87fee87b864cd0fa9a51fdc9 not found: ID does not exist" Jul 28 13:49:45.297354 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.297333 2550 scope.go:117] "RemoveContainer" containerID="c1e9a064e55b8786635e4087071da82b1c70de20cd1324ce6b43a4552897a8c6" Jul 28 13:49:45.297577 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.297556 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1e9a064e55b8786635e4087071da82b1c70de20cd1324ce6b43a4552897a8c6"} err="failed to get container status \"c1e9a064e55b8786635e4087071da82b1c70de20cd1324ce6b43a4552897a8c6\": rpc error: code = NotFound desc = could not find container \"c1e9a064e55b8786635e4087071da82b1c70de20cd1324ce6b43a4552897a8c6\": container with ID starting with c1e9a064e55b8786635e4087071da82b1c70de20cd1324ce6b43a4552897a8c6 not found: ID does not exist" Jul 28 13:49:45.297646 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.297578 2550 scope.go:117] "RemoveContainer" containerID="d2f997e70df7b051cf373acb4f9509a9fbcdeb79e00706d853bf4ac143f723dd" Jul 28 13:49:45.297785 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.297764 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2f997e70df7b051cf373acb4f9509a9fbcdeb79e00706d853bf4ac143f723dd"} err="failed to get container status \"d2f997e70df7b051cf373acb4f9509a9fbcdeb79e00706d853bf4ac143f723dd\": rpc error: code = NotFound desc = could not find container \"d2f997e70df7b051cf373acb4f9509a9fbcdeb79e00706d853bf4ac143f723dd\": container with ID starting with d2f997e70df7b051cf373acb4f9509a9fbcdeb79e00706d853bf4ac143f723dd not found: ID does not exist" Jul 28 13:49:45.297826 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.297786 2550 scope.go:117] "RemoveContainer" containerID="b17f6644d14101b3fd843c5b062abf07c2be9074aba09899390e2fa69080d02b" Jul 28 13:49:45.297988 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.297971 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b17f6644d14101b3fd843c5b062abf07c2be9074aba09899390e2fa69080d02b"} err="failed to get container status \"b17f6644d14101b3fd843c5b062abf07c2be9074aba09899390e2fa69080d02b\": rpc error: code = NotFound desc = could not find container \"b17f6644d14101b3fd843c5b062abf07c2be9074aba09899390e2fa69080d02b\": container with ID starting with b17f6644d14101b3fd843c5b062abf07c2be9074aba09899390e2fa69080d02b not found: ID does not exist" Jul 28 13:49:45.298053 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.297989 2550 scope.go:117] "RemoveContainer" containerID="e02897857e885176cff1ccc7dcea3a1f61f3b608db349355ee9eec4d51110a28" Jul 28 13:49:45.298245 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.298228 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e02897857e885176cff1ccc7dcea3a1f61f3b608db349355ee9eec4d51110a28"} err="failed to get container status \"e02897857e885176cff1ccc7dcea3a1f61f3b608db349355ee9eec4d51110a28\": rpc error: code = NotFound desc = could not find container \"e02897857e885176cff1ccc7dcea3a1f61f3b608db349355ee9eec4d51110a28\": container with ID starting with e02897857e885176cff1ccc7dcea3a1f61f3b608db349355ee9eec4d51110a28 not found: ID does not exist" Jul 28 13:49:45.298295 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.298246 2550 scope.go:117] "RemoveContainer" containerID="19e39320428fea534d2d3d31970c3ca4143d49e5408aa7154cc367b34daee637" Jul 28 13:49:45.298463 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.298446 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19e39320428fea534d2d3d31970c3ca4143d49e5408aa7154cc367b34daee637"} err="failed to get container status \"19e39320428fea534d2d3d31970c3ca4143d49e5408aa7154cc367b34daee637\": rpc error: code = NotFound desc = could not find container \"19e39320428fea534d2d3d31970c3ca4143d49e5408aa7154cc367b34daee637\": container with ID starting with 19e39320428fea534d2d3d31970c3ca4143d49e5408aa7154cc367b34daee637 not found: ID does not exist" Jul 28 13:49:45.298515 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.298464 2550 scope.go:117] "RemoveContainer" containerID="4bfe0b879d58e16a27da54b68097023a039c1caf8e893646119e34553f799fe1" Jul 28 13:49:45.298661 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.298639 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bfe0b879d58e16a27da54b68097023a039c1caf8e893646119e34553f799fe1"} err="failed to get container status \"4bfe0b879d58e16a27da54b68097023a039c1caf8e893646119e34553f799fe1\": rpc error: code = NotFound desc = could not find container \"4bfe0b879d58e16a27da54b68097023a039c1caf8e893646119e34553f799fe1\": container with ID starting with 4bfe0b879d58e16a27da54b68097023a039c1caf8e893646119e34553f799fe1 not found: ID does not exist" Jul 28 13:49:45.298709 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.298660 2550 scope.go:117] "RemoveContainer" containerID="d3edce344422b034828e50de178c751a65de672e87fee87b864cd0fa9a51fdc9" Jul 28 13:49:45.298896 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.298878 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3edce344422b034828e50de178c751a65de672e87fee87b864cd0fa9a51fdc9"} err="failed to get container status \"d3edce344422b034828e50de178c751a65de672e87fee87b864cd0fa9a51fdc9\": rpc error: code = NotFound desc = could not find container \"d3edce344422b034828e50de178c751a65de672e87fee87b864cd0fa9a51fdc9\": container with ID starting with d3edce344422b034828e50de178c751a65de672e87fee87b864cd0fa9a51fdc9 not found: ID does not exist" Jul 28 13:49:45.309445 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.309411 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mn8nb\" (UniqueName: \"kubernetes.io/projected/d43548cb-241a-490a-a94f-7c08efe068cc-kube-api-access-mn8nb\") on node \"ip-10-0-141-63.ec2.internal\" DevicePath \"\"" Jul 28 13:49:45.309445 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.309445 2550 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d43548cb-241a-490a-a94f-7c08efe068cc-metrics-client-ca\") on node \"ip-10-0-141-63.ec2.internal\" DevicePath \"\"" Jul 28 13:49:45.309641 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.309457 2550 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d43548cb-241a-490a-a94f-7c08efe068cc-config-out\") on node \"ip-10-0-141-63.ec2.internal\" DevicePath \"\"" Jul 28 13:49:45.309641 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.309466 2550 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d43548cb-241a-490a-a94f-7c08efe068cc-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-141-63.ec2.internal\" DevicePath \"\"" Jul 28 13:49:45.309641 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.309475 2550 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-141-63.ec2.internal\" DevicePath \"\"" Jul 28 13:49:45.309641 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.309484 2550 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d43548cb-241a-490a-a94f-7c08efe068cc-alertmanager-main-db\") on node \"ip-10-0-141-63.ec2.internal\" DevicePath \"\"" Jul 28 13:49:45.309641 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.309493 2550 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-141-63.ec2.internal\" DevicePath \"\"" Jul 28 13:49:45.309641 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.309501 2550 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-config-volume\") on node \"ip-10-0-141-63.ec2.internal\" DevicePath \"\"" Jul 28 13:49:45.309641 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.309509 2550 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-cluster-tls-config\") on node \"ip-10-0-141-63.ec2.internal\" DevicePath \"\"" Jul 28 13:49:45.309641 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.309518 2550 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-secret-alertmanager-main-tls\") on node \"ip-10-0-141-63.ec2.internal\" DevicePath \"\"" Jul 28 13:49:45.309641 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.309527 2550 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d43548cb-241a-490a-a94f-7c08efe068cc-tls-assets\") on node \"ip-10-0-141-63.ec2.internal\" DevicePath \"\"" Jul 28 13:49:45.309641 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.309536 2550 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-141-63.ec2.internal\" DevicePath \"\"" Jul 28 13:49:45.309641 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.309544 2550 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d43548cb-241a-490a-a94f-7c08efe068cc-web-config\") on node \"ip-10-0-141-63.ec2.internal\" DevicePath \"\"" Jul 28 13:49:45.318404 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.318369 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 13:49:45.318720 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.318707 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d43548cb-241a-490a-a94f-7c08efe068cc" containerName="config-reloader" Jul 28 13:49:45.318759 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.318721 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="d43548cb-241a-490a-a94f-7c08efe068cc" containerName="config-reloader" Jul 28 13:49:45.318759 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.318731 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d43548cb-241a-490a-a94f-7c08efe068cc" containerName="kube-rbac-proxy-web" Jul 28 13:49:45.318759 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.318737 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="d43548cb-241a-490a-a94f-7c08efe068cc" containerName="kube-rbac-proxy-web" Jul 28 13:49:45.318759 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.318748 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d43548cb-241a-490a-a94f-7c08efe068cc" containerName="prom-label-proxy" Jul 28 13:49:45.318759 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.318753 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="d43548cb-241a-490a-a94f-7c08efe068cc" containerName="prom-label-proxy" Jul 28 13:49:45.318906 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.318768 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d43548cb-241a-490a-a94f-7c08efe068cc" containerName="init-config-reloader" Jul 28 13:49:45.318906 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.318774 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="d43548cb-241a-490a-a94f-7c08efe068cc" containerName="init-config-reloader" Jul 28 13:49:45.318906 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.318781 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d43548cb-241a-490a-a94f-7c08efe068cc" containerName="kube-rbac-proxy-metric" Jul 28 13:49:45.318906 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.318786 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="d43548cb-241a-490a-a94f-7c08efe068cc" containerName="kube-rbac-proxy-metric" Jul 28 13:49:45.318906 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.318799 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d43548cb-241a-490a-a94f-7c08efe068cc" containerName="alertmanager" Jul 28 13:49:45.318906 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.318805 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="d43548cb-241a-490a-a94f-7c08efe068cc" containerName="alertmanager" Jul 28 13:49:45.318906 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.318810 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d43548cb-241a-490a-a94f-7c08efe068cc" containerName="kube-rbac-proxy" Jul 28 13:49:45.318906 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.318815 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="d43548cb-241a-490a-a94f-7c08efe068cc" containerName="kube-rbac-proxy" Jul 28 13:49:45.318906 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.318864 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="d43548cb-241a-490a-a94f-7c08efe068cc" containerName="prom-label-proxy" Jul 28 13:49:45.318906 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.318872 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="d43548cb-241a-490a-a94f-7c08efe068cc" containerName="config-reloader" Jul 28 13:49:45.318906 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.318878 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="d43548cb-241a-490a-a94f-7c08efe068cc" containerName="kube-rbac-proxy-web" Jul 28 13:49:45.318906 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.318883 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="d43548cb-241a-490a-a94f-7c08efe068cc" containerName="kube-rbac-proxy-metric" Jul 28 13:49:45.318906 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.318890 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="d43548cb-241a-490a-a94f-7c08efe068cc" containerName="alertmanager" Jul 28 13:49:45.318906 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.318896 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="d43548cb-241a-490a-a94f-7c08efe068cc" containerName="kube-rbac-proxy" Jul 28 13:49:45.322776 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.322758 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.326582 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.326550 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 28 13:49:45.326747 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.326598 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 28 13:49:45.326852 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.326830 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 28 13:49:45.326974 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.326850 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 28 13:49:45.326974 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.326860 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 28 13:49:45.326974 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.326910 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-mn46k\"" Jul 28 13:49:45.326974 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.326968 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 28 13:49:45.327201 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.327099 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 28 13:49:45.327201 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.327110 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 28 13:49:45.330921 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.330899 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 28 13:49:45.341913 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.341886 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 13:49:45.410175 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.410134 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.410175 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.410171 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-config-volume\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.410434 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.410190 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.410434 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.410211 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-web-config\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.410434 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.410323 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.410434 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.410358 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95zm5\" (UniqueName: \"kubernetes.io/projected/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-kube-api-access-95zm5\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.410434 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.410412 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.410707 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.410442 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.410707 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.410465 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.410707 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.410507 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.410707 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.410559 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.410707 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.410582 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.410707 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.410628 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-config-out\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.511795 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.511709 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.511795 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.511748 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.511795 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.511772 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-config-out\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.511795 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.511797 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.512156 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.511827 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-config-volume\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.512156 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.511852 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.512156 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.511881 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-web-config\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.512156 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.511948 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.512156 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.511973 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95zm5\" (UniqueName: \"kubernetes.io/projected/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-kube-api-access-95zm5\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.512156 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.511998 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.512156 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.512024 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.512156 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.512051 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.512156 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.512083 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.513168 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.512815 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.513168 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.513083 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.514976 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.514951 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-config-out\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.515086 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.515003 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-web-config\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.515086 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.515031 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-config-volume\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.515086 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.515055 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.515236 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.515218 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.515293 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.515238 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.515293 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.515263 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.515585 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.515567 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.515945 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.515929 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.516833 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.516817 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.521831 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.521812 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95zm5\" (UniqueName: \"kubernetes.io/projected/8b900ed2-0eb3-4945-b487-0db3fb0d2a34-kube-api-access-95zm5\") pod \"alertmanager-main-0\" (UID: \"8b900ed2-0eb3-4945-b487-0db3fb0d2a34\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.633808 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.633778 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 13:49:45.792295 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:45.792259 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 13:49:45.793298 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:49:45.793275 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b900ed2_0eb3_4945_b487_0db3fb0d2a34.slice/crio-cec4c6d30c0aa5d1776f5bf5446efa4d7615038cc342889d07a29cad44b52b1e WatchSource:0}: Error finding container cec4c6d30c0aa5d1776f5bf5446efa4d7615038cc342889d07a29cad44b52b1e: Status 404 returned error can't find the container with id cec4c6d30c0aa5d1776f5bf5446efa4d7615038cc342889d07a29cad44b52b1e Jul 28 13:49:46.250734 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:46.250698 2550 generic.go:356] "Generic (PLEG): container finished" podID="8b900ed2-0eb3-4945-b487-0db3fb0d2a34" containerID="228194a4042d02c1a5fd8dcab52901c1ebdd6fa0c6230fa97f84c8879016bf4d" exitCode=0 Jul 28 13:49:46.251123 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:46.250780 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8b900ed2-0eb3-4945-b487-0db3fb0d2a34","Type":"ContainerDied","Data":"228194a4042d02c1a5fd8dcab52901c1ebdd6fa0c6230fa97f84c8879016bf4d"} Jul 28 13:49:46.251123 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:46.250856 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8b900ed2-0eb3-4945-b487-0db3fb0d2a34","Type":"ContainerStarted","Data":"cec4c6d30c0aa5d1776f5bf5446efa4d7615038cc342889d07a29cad44b52b1e"} Jul 28 13:49:46.477653 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:46.477627 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d43548cb-241a-490a-a94f-7c08efe068cc" path="/var/lib/kubelet/pods/d43548cb-241a-490a-a94f-7c08efe068cc/volumes" Jul 28 13:49:47.260602 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:47.260562 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8b900ed2-0eb3-4945-b487-0db3fb0d2a34","Type":"ContainerStarted","Data":"b84640f8eaf677e76b82643f085c55cd879078eddf2f36a626d89837f07b8491"} Jul 28 13:49:47.260968 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:47.260632 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8b900ed2-0eb3-4945-b487-0db3fb0d2a34","Type":"ContainerStarted","Data":"08cc61b7d406ad4bb2d56c6fd99d443e2580790ddf60a8248270a10dccae8e3d"} Jul 28 13:49:47.260968 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:47.260662 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8b900ed2-0eb3-4945-b487-0db3fb0d2a34","Type":"ContainerStarted","Data":"27a28d52fe2c33b91736b17df0fd8d9dd6fd9ca0f23e28a89d45cb3687eaf2aa"} Jul 28 13:49:47.260968 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:47.260685 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8b900ed2-0eb3-4945-b487-0db3fb0d2a34","Type":"ContainerStarted","Data":"3a2dbd65889886a39576dad45a952b86155bdc015e1c4fc136f2a96190e22e57"} Jul 28 13:49:47.260968 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:47.260710 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8b900ed2-0eb3-4945-b487-0db3fb0d2a34","Type":"ContainerStarted","Data":"8f396055c5284b57425a94b05c7b42a99ef9b1880b368658b9f8bbcdf5d14f27"} Jul 28 13:49:47.260968 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:47.260734 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8b900ed2-0eb3-4945-b487-0db3fb0d2a34","Type":"ContainerStarted","Data":"8c345a11e2d3384ac03488501e900831850d3065fefff74b50e2a08c7cbe74dd"} Jul 28 13:49:47.292936 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:47.292867 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.276923508 podStartE2EDuration="2.292845413s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.921905ms" isStatefulPod=false podCreationTimestamp="2026-07-28 13:49:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:49:47.291923806 +0000 UTC m=+141.427120698" watchObservedRunningTime="2026-07-28 13:49:47.292845413 +0000 UTC m=+141.428042305" Jul 28 13:49:48.144328 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.144294 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 13:49:48.144661 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.144629 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" containerName="prometheus" containerID="cri-o://b0f3145a81cc40b101d3797e3ccb2968894ee11d801f5d32fc338d47d802f1fc" gracePeriod=600 Jul 28 13:49:48.144751 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.144657 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" containerName="thanos-sidecar" containerID="cri-o://672b9bb96f6aabbab28f7255fb823be808212cf477650976f77ba47d7dc55fdf" gracePeriod=600 Jul 28 13:49:48.144751 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.144672 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" containerName="kube-rbac-proxy-web" containerID="cri-o://0b3336c901521ece0f2cc46d92ebe011025853bf00ca345f370523466da91622" gracePeriod=600 Jul 28 13:49:48.144751 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.144677 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" containerName="config-reloader" containerID="cri-o://091186c4a45bd814266ac30e567725721c206e48735026863abe79f7742e2101" gracePeriod=600 Jul 28 13:49:48.144751 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.144692 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" containerName="kube-rbac-proxy-thanos" containerID="cri-o://4bda1c363205d9aa2c8da95e0bebd585c97333b0529628384da036e5bc789d6a" gracePeriod=600 Jul 28 13:49:48.144938 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.144639 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" containerName="kube-rbac-proxy" containerID="cri-o://ba84ae31980f080e56ec77a7865406afa2bfe3175cf19a429e998c68acfa325d" gracePeriod=600 Jul 28 13:49:48.277376 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.277338 2550 generic.go:356] "Generic (PLEG): container finished" podID="cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" containerID="ba84ae31980f080e56ec77a7865406afa2bfe3175cf19a429e998c68acfa325d" exitCode=0 Jul 28 13:49:48.277733 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.277379 2550 generic.go:356] "Generic (PLEG): container finished" podID="cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" containerID="0b3336c901521ece0f2cc46d92ebe011025853bf00ca345f370523466da91622" exitCode=0 Jul 28 13:49:48.277733 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.277421 2550 generic.go:356] "Generic (PLEG): container finished" podID="cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" containerID="672b9bb96f6aabbab28f7255fb823be808212cf477650976f77ba47d7dc55fdf" exitCode=0 Jul 28 13:49:48.277733 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.277433 2550 generic.go:356] "Generic (PLEG): container finished" podID="cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" containerID="b0f3145a81cc40b101d3797e3ccb2968894ee11d801f5d32fc338d47d802f1fc" exitCode=0 Jul 28 13:49:48.277733 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.277453 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d","Type":"ContainerDied","Data":"ba84ae31980f080e56ec77a7865406afa2bfe3175cf19a429e998c68acfa325d"} Jul 28 13:49:48.277733 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.277598 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d","Type":"ContainerDied","Data":"0b3336c901521ece0f2cc46d92ebe011025853bf00ca345f370523466da91622"} Jul 28 13:49:48.277733 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.277643 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d","Type":"ContainerDied","Data":"672b9bb96f6aabbab28f7255fb823be808212cf477650976f77ba47d7dc55fdf"} Jul 28 13:49:48.277733 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.277684 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d","Type":"ContainerDied","Data":"b0f3145a81cc40b101d3797e3ccb2968894ee11d801f5d32fc338d47d802f1fc"} Jul 28 13:49:48.408171 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.408146 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:48.439919 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.439884 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-thanos-prometheus-http-client-file\") pod \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " Jul 28 13:49:48.440092 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.439933 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hl8s8\" (UniqueName: \"kubernetes.io/projected/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-kube-api-access-hl8s8\") pod \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " Jul 28 13:49:48.440092 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.439970 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-prometheus-k8s-db\") pod \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " Jul 28 13:49:48.440092 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.439998 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-config\") pod \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " Jul 28 13:49:48.440092 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.440026 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-secret-kube-rbac-proxy\") pod \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " Jul 28 13:49:48.440092 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.440068 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-configmap-metrics-client-ca\") pod \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " Jul 28 13:49:48.440357 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.440095 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-secret-grpc-tls\") pod \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " Jul 28 13:49:48.440357 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.440119 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " Jul 28 13:49:48.440357 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.440152 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-prometheus-k8s-rulefiles-1\") pod \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " Jul 28 13:49:48.440357 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.440182 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-config-out\") pod \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " Jul 28 13:49:48.440357 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.440210 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-configmap-serving-certs-ca-bundle\") pod \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " Jul 28 13:49:48.440357 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.440242 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-secret-prometheus-k8s-tls\") pod \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " Jul 28 13:49:48.440357 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.440278 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-configmap-kubelet-serving-ca-bundle\") pod \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " Jul 28 13:49:48.440357 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.440307 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-prometheus-k8s-rulefiles-2\") pod \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " Jul 28 13:49:48.440357 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.440330 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-secret-metrics-client-certs\") pod \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " Jul 28 13:49:48.440357 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.440354 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-prometheus-trusted-ca-bundle\") pod \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " Jul 28 13:49:48.440877 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.440420 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-web-config\") pod \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " Jul 28 13:49:48.440877 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.440469 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " Jul 28 13:49:48.440877 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.440517 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-prometheus-k8s-rulefiles-0\") pod \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " Jul 28 13:49:48.440877 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.440541 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-tls-assets\") pod \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\" (UID: \"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d\") " Jul 28 13:49:48.440877 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.440557 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" (UID: "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 13:49:48.440877 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.440816 2550 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-configmap-metrics-client-ca\") on node \"ip-10-0-141-63.ec2.internal\" DevicePath \"\"" Jul 28 13:49:48.442285 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.441377 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" (UID: "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 13:49:48.442285 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.441722 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" (UID: "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 13:49:48.442285 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.441922 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" (UID: "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 13:49:48.442574 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.442456 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" (UID: "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 13:49:48.442909 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.442880 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" (UID: "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 13:49:48.443452 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.443324 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" (UID: "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 13:49:48.444962 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.444930 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" (UID: "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 13:49:48.445059 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.444970 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-config" (OuterVolumeSpecName: "config") pod "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" (UID: "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 13:49:48.446950 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.446911 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" (UID: "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 13:49:48.447717 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.447662 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" (UID: "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 13:49:48.447816 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.447775 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" (UID: "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 13:49:48.448120 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.448087 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-config-out" (OuterVolumeSpecName: "config-out") pod "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" (UID: "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 13:49:48.448462 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.448379 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" (UID: "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 13:49:48.448817 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.448778 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-kube-api-access-hl8s8" (OuterVolumeSpecName: "kube-api-access-hl8s8") pod "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" (UID: "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d"). InnerVolumeSpecName "kube-api-access-hl8s8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 13:49:48.448817 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.448798 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" (UID: "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 13:49:48.449288 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.449133 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" (UID: "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 13:49:48.449288 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.449263 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" (UID: "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 13:49:48.451952 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.451918 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" (UID: "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 13:49:48.462199 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.462166 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-web-config" (OuterVolumeSpecName: "web-config") pod "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" (UID: "cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 13:49:48.542454 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.542415 2550 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-141-63.ec2.internal\" DevicePath \"\"" Jul 28 13:49:48.542678 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.542475 2550 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-tls-assets\") on node \"ip-10-0-141-63.ec2.internal\" DevicePath \"\"" Jul 28 13:49:48.542678 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.542494 2550 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-thanos-prometheus-http-client-file\") on node \"ip-10-0-141-63.ec2.internal\" DevicePath \"\"" Jul 28 13:49:48.542678 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.542510 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hl8s8\" (UniqueName: \"kubernetes.io/projected/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-kube-api-access-hl8s8\") on node \"ip-10-0-141-63.ec2.internal\" DevicePath \"\"" Jul 28 13:49:48.542678 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.542523 2550 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-prometheus-k8s-db\") on node \"ip-10-0-141-63.ec2.internal\" DevicePath \"\"" Jul 28 13:49:48.542678 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.542536 2550 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-config\") on node \"ip-10-0-141-63.ec2.internal\" DevicePath \"\"" Jul 28 13:49:48.542678 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.542548 2550 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-secret-kube-rbac-proxy\") on node \"ip-10-0-141-63.ec2.internal\" DevicePath \"\"" Jul 28 13:49:48.542678 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.542557 2550 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-secret-grpc-tls\") on node \"ip-10-0-141-63.ec2.internal\" DevicePath \"\"" Jul 28 13:49:48.542678 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.542566 2550 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-141-63.ec2.internal\" DevicePath \"\"" Jul 28 13:49:48.542678 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.542576 2550 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-141-63.ec2.internal\" DevicePath \"\"" Jul 28 13:49:48.542678 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.542585 2550 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-config-out\") on node \"ip-10-0-141-63.ec2.internal\" DevicePath \"\"" Jul 28 13:49:48.542678 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.542593 2550 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-141-63.ec2.internal\" DevicePath \"\"" Jul 28 13:49:48.542678 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.542601 2550 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-secret-prometheus-k8s-tls\") on node \"ip-10-0-141-63.ec2.internal\" DevicePath \"\"" Jul 28 13:49:48.542678 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.542610 2550 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-141-63.ec2.internal\" DevicePath \"\"" Jul 28 13:49:48.542678 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.542620 2550 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-141-63.ec2.internal\" DevicePath \"\"" Jul 28 13:49:48.542678 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.542628 2550 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-secret-metrics-client-certs\") on node \"ip-10-0-141-63.ec2.internal\" DevicePath \"\"" Jul 28 13:49:48.542678 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.542636 2550 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-prometheus-trusted-ca-bundle\") on node \"ip-10-0-141-63.ec2.internal\" DevicePath \"\"" Jul 28 13:49:48.542678 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.542644 2550 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-web-config\") on node \"ip-10-0-141-63.ec2.internal\" DevicePath \"\"" Jul 28 13:49:48.542678 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:48.542661 2550 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-141-63.ec2.internal\" DevicePath \"\"" Jul 28 13:49:49.284846 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.284812 2550 generic.go:356] "Generic (PLEG): container finished" podID="cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" containerID="4bda1c363205d9aa2c8da95e0bebd585c97333b0529628384da036e5bc789d6a" exitCode=0 Jul 28 13:49:49.284846 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.284838 2550 generic.go:356] "Generic (PLEG): container finished" podID="cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" containerID="091186c4a45bd814266ac30e567725721c206e48735026863abe79f7742e2101" exitCode=0 Jul 28 13:49:49.285440 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.284867 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d","Type":"ContainerDied","Data":"4bda1c363205d9aa2c8da95e0bebd585c97333b0529628384da036e5bc789d6a"} Jul 28 13:49:49.285440 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.284885 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.285440 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.284940 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d","Type":"ContainerDied","Data":"091186c4a45bd814266ac30e567725721c206e48735026863abe79f7742e2101"} Jul 28 13:49:49.285440 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.284985 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d","Type":"ContainerDied","Data":"d992fc72e69f52c7315350e8f29b8ec7012a92d265c5af66be4d6fc76389f4a9"} Jul 28 13:49:49.285440 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.285018 2550 scope.go:117] "RemoveContainer" containerID="4bda1c363205d9aa2c8da95e0bebd585c97333b0529628384da036e5bc789d6a" Jul 28 13:49:49.293425 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.293406 2550 scope.go:117] "RemoveContainer" containerID="ba84ae31980f080e56ec77a7865406afa2bfe3175cf19a429e998c68acfa325d" Jul 28 13:49:49.301518 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.301492 2550 scope.go:117] "RemoveContainer" containerID="0b3336c901521ece0f2cc46d92ebe011025853bf00ca345f370523466da91622" Jul 28 13:49:49.309212 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.309192 2550 scope.go:117] "RemoveContainer" containerID="672b9bb96f6aabbab28f7255fb823be808212cf477650976f77ba47d7dc55fdf" Jul 28 13:49:49.312272 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.312245 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 13:49:49.317162 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.317137 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 13:49:49.318499 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.318478 2550 scope.go:117] "RemoveContainer" containerID="091186c4a45bd814266ac30e567725721c206e48735026863abe79f7742e2101" Jul 28 13:49:49.325811 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.325789 2550 scope.go:117] "RemoveContainer" containerID="b0f3145a81cc40b101d3797e3ccb2968894ee11d801f5d32fc338d47d802f1fc" Jul 28 13:49:49.333337 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.333316 2550 scope.go:117] "RemoveContainer" containerID="727c52f4873cc6b2b374041bd49ccdb19cf16ea52fbacd85505e857623c03963" Jul 28 13:49:49.341069 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.341049 2550 scope.go:117] "RemoveContainer" containerID="4bda1c363205d9aa2c8da95e0bebd585c97333b0529628384da036e5bc789d6a" Jul 28 13:49:49.341385 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:49:49.341364 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bda1c363205d9aa2c8da95e0bebd585c97333b0529628384da036e5bc789d6a\": container with ID starting with 4bda1c363205d9aa2c8da95e0bebd585c97333b0529628384da036e5bc789d6a not found: ID does not exist" containerID="4bda1c363205d9aa2c8da95e0bebd585c97333b0529628384da036e5bc789d6a" Jul 28 13:49:49.341460 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.341418 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bda1c363205d9aa2c8da95e0bebd585c97333b0529628384da036e5bc789d6a"} err="failed to get container status \"4bda1c363205d9aa2c8da95e0bebd585c97333b0529628384da036e5bc789d6a\": rpc error: code = NotFound desc = could not find container \"4bda1c363205d9aa2c8da95e0bebd585c97333b0529628384da036e5bc789d6a\": container with ID starting with 4bda1c363205d9aa2c8da95e0bebd585c97333b0529628384da036e5bc789d6a not found: ID does not exist" Jul 28 13:49:49.341460 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.341439 2550 scope.go:117] "RemoveContainer" containerID="ba84ae31980f080e56ec77a7865406afa2bfe3175cf19a429e998c68acfa325d" Jul 28 13:49:49.341690 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:49:49.341673 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba84ae31980f080e56ec77a7865406afa2bfe3175cf19a429e998c68acfa325d\": container with ID starting with ba84ae31980f080e56ec77a7865406afa2bfe3175cf19a429e998c68acfa325d not found: ID does not exist" containerID="ba84ae31980f080e56ec77a7865406afa2bfe3175cf19a429e998c68acfa325d" Jul 28 13:49:49.341810 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.341699 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba84ae31980f080e56ec77a7865406afa2bfe3175cf19a429e998c68acfa325d"} err="failed to get container status \"ba84ae31980f080e56ec77a7865406afa2bfe3175cf19a429e998c68acfa325d\": rpc error: code = NotFound desc = could not find container \"ba84ae31980f080e56ec77a7865406afa2bfe3175cf19a429e998c68acfa325d\": container with ID starting with ba84ae31980f080e56ec77a7865406afa2bfe3175cf19a429e998c68acfa325d not found: ID does not exist" Jul 28 13:49:49.341810 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.341723 2550 scope.go:117] "RemoveContainer" containerID="0b3336c901521ece0f2cc46d92ebe011025853bf00ca345f370523466da91622" Jul 28 13:49:49.341952 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:49:49.341935 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b3336c901521ece0f2cc46d92ebe011025853bf00ca345f370523466da91622\": container with ID starting with 0b3336c901521ece0f2cc46d92ebe011025853bf00ca345f370523466da91622 not found: ID does not exist" containerID="0b3336c901521ece0f2cc46d92ebe011025853bf00ca345f370523466da91622" Jul 28 13:49:49.341989 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.341956 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b3336c901521ece0f2cc46d92ebe011025853bf00ca345f370523466da91622"} err="failed to get container status \"0b3336c901521ece0f2cc46d92ebe011025853bf00ca345f370523466da91622\": rpc error: code = NotFound desc = could not find container \"0b3336c901521ece0f2cc46d92ebe011025853bf00ca345f370523466da91622\": container with ID starting with 0b3336c901521ece0f2cc46d92ebe011025853bf00ca345f370523466da91622 not found: ID does not exist" Jul 28 13:49:49.341989 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.341971 2550 scope.go:117] "RemoveContainer" containerID="672b9bb96f6aabbab28f7255fb823be808212cf477650976f77ba47d7dc55fdf" Jul 28 13:49:49.342202 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:49:49.342186 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"672b9bb96f6aabbab28f7255fb823be808212cf477650976f77ba47d7dc55fdf\": container with ID starting with 672b9bb96f6aabbab28f7255fb823be808212cf477650976f77ba47d7dc55fdf not found: ID does not exist" containerID="672b9bb96f6aabbab28f7255fb823be808212cf477650976f77ba47d7dc55fdf" Jul 28 13:49:49.342245 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.342204 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"672b9bb96f6aabbab28f7255fb823be808212cf477650976f77ba47d7dc55fdf"} err="failed to get container status \"672b9bb96f6aabbab28f7255fb823be808212cf477650976f77ba47d7dc55fdf\": rpc error: code = NotFound desc = could not find container \"672b9bb96f6aabbab28f7255fb823be808212cf477650976f77ba47d7dc55fdf\": container with ID starting with 672b9bb96f6aabbab28f7255fb823be808212cf477650976f77ba47d7dc55fdf not found: ID does not exist" Jul 28 13:49:49.342245 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.342217 2550 scope.go:117] "RemoveContainer" containerID="091186c4a45bd814266ac30e567725721c206e48735026863abe79f7742e2101" Jul 28 13:49:49.342528 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:49:49.342512 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"091186c4a45bd814266ac30e567725721c206e48735026863abe79f7742e2101\": container with ID starting with 091186c4a45bd814266ac30e567725721c206e48735026863abe79f7742e2101 not found: ID does not exist" containerID="091186c4a45bd814266ac30e567725721c206e48735026863abe79f7742e2101" Jul 28 13:49:49.342586 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.342531 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"091186c4a45bd814266ac30e567725721c206e48735026863abe79f7742e2101"} err="failed to get container status \"091186c4a45bd814266ac30e567725721c206e48735026863abe79f7742e2101\": rpc error: code = NotFound desc = could not find container \"091186c4a45bd814266ac30e567725721c206e48735026863abe79f7742e2101\": container with ID starting with 091186c4a45bd814266ac30e567725721c206e48735026863abe79f7742e2101 not found: ID does not exist" Jul 28 13:49:49.342586 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.342544 2550 scope.go:117] "RemoveContainer" containerID="b0f3145a81cc40b101d3797e3ccb2968894ee11d801f5d32fc338d47d802f1fc" Jul 28 13:49:49.342836 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:49:49.342819 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0f3145a81cc40b101d3797e3ccb2968894ee11d801f5d32fc338d47d802f1fc\": container with ID starting with b0f3145a81cc40b101d3797e3ccb2968894ee11d801f5d32fc338d47d802f1fc not found: ID does not exist" containerID="b0f3145a81cc40b101d3797e3ccb2968894ee11d801f5d32fc338d47d802f1fc" Jul 28 13:49:49.342932 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.342843 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0f3145a81cc40b101d3797e3ccb2968894ee11d801f5d32fc338d47d802f1fc"} err="failed to get container status \"b0f3145a81cc40b101d3797e3ccb2968894ee11d801f5d32fc338d47d802f1fc\": rpc error: code = NotFound desc = could not find container \"b0f3145a81cc40b101d3797e3ccb2968894ee11d801f5d32fc338d47d802f1fc\": container with ID starting with b0f3145a81cc40b101d3797e3ccb2968894ee11d801f5d32fc338d47d802f1fc not found: ID does not exist" Jul 28 13:49:49.342998 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.342935 2550 scope.go:117] "RemoveContainer" containerID="727c52f4873cc6b2b374041bd49ccdb19cf16ea52fbacd85505e857623c03963" Jul 28 13:49:49.343480 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:49:49.343340 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"727c52f4873cc6b2b374041bd49ccdb19cf16ea52fbacd85505e857623c03963\": container with ID starting with 727c52f4873cc6b2b374041bd49ccdb19cf16ea52fbacd85505e857623c03963 not found: ID does not exist" containerID="727c52f4873cc6b2b374041bd49ccdb19cf16ea52fbacd85505e857623c03963" Jul 28 13:49:49.343480 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.343458 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"727c52f4873cc6b2b374041bd49ccdb19cf16ea52fbacd85505e857623c03963"} err="failed to get container status \"727c52f4873cc6b2b374041bd49ccdb19cf16ea52fbacd85505e857623c03963\": rpc error: code = NotFound desc = could not find container \"727c52f4873cc6b2b374041bd49ccdb19cf16ea52fbacd85505e857623c03963\": container with ID starting with 727c52f4873cc6b2b374041bd49ccdb19cf16ea52fbacd85505e857623c03963 not found: ID does not exist" Jul 28 13:49:49.343721 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.343487 2550 scope.go:117] "RemoveContainer" containerID="4bda1c363205d9aa2c8da95e0bebd585c97333b0529628384da036e5bc789d6a" Jul 28 13:49:49.343817 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.343789 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bda1c363205d9aa2c8da95e0bebd585c97333b0529628384da036e5bc789d6a"} err="failed to get container status \"4bda1c363205d9aa2c8da95e0bebd585c97333b0529628384da036e5bc789d6a\": rpc error: code = NotFound desc = could not find container \"4bda1c363205d9aa2c8da95e0bebd585c97333b0529628384da036e5bc789d6a\": container with ID starting with 4bda1c363205d9aa2c8da95e0bebd585c97333b0529628384da036e5bc789d6a not found: ID does not exist" Jul 28 13:49:49.343817 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.343816 2550 scope.go:117] "RemoveContainer" containerID="ba84ae31980f080e56ec77a7865406afa2bfe3175cf19a429e998c68acfa325d" Jul 28 13:49:49.344156 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.344129 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba84ae31980f080e56ec77a7865406afa2bfe3175cf19a429e998c68acfa325d"} err="failed to get container status \"ba84ae31980f080e56ec77a7865406afa2bfe3175cf19a429e998c68acfa325d\": rpc error: code = NotFound desc = could not find container \"ba84ae31980f080e56ec77a7865406afa2bfe3175cf19a429e998c68acfa325d\": container with ID starting with ba84ae31980f080e56ec77a7865406afa2bfe3175cf19a429e998c68acfa325d not found: ID does not exist" Jul 28 13:49:49.344156 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.344156 2550 scope.go:117] "RemoveContainer" containerID="0b3336c901521ece0f2cc46d92ebe011025853bf00ca345f370523466da91622" Jul 28 13:49:49.344460 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.344436 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b3336c901521ece0f2cc46d92ebe011025853bf00ca345f370523466da91622"} err="failed to get container status \"0b3336c901521ece0f2cc46d92ebe011025853bf00ca345f370523466da91622\": rpc error: code = NotFound desc = could not find container \"0b3336c901521ece0f2cc46d92ebe011025853bf00ca345f370523466da91622\": container with ID starting with 0b3336c901521ece0f2cc46d92ebe011025853bf00ca345f370523466da91622 not found: ID does not exist" Jul 28 13:49:49.344460 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.344459 2550 scope.go:117] "RemoveContainer" containerID="672b9bb96f6aabbab28f7255fb823be808212cf477650976f77ba47d7dc55fdf" Jul 28 13:49:49.344723 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.344706 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"672b9bb96f6aabbab28f7255fb823be808212cf477650976f77ba47d7dc55fdf"} err="failed to get container status \"672b9bb96f6aabbab28f7255fb823be808212cf477650976f77ba47d7dc55fdf\": rpc error: code = NotFound desc = could not find container \"672b9bb96f6aabbab28f7255fb823be808212cf477650976f77ba47d7dc55fdf\": container with ID starting with 672b9bb96f6aabbab28f7255fb823be808212cf477650976f77ba47d7dc55fdf not found: ID does not exist" Jul 28 13:49:49.344782 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.344725 2550 scope.go:117] "RemoveContainer" containerID="091186c4a45bd814266ac30e567725721c206e48735026863abe79f7742e2101" Jul 28 13:49:49.344931 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.344914 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"091186c4a45bd814266ac30e567725721c206e48735026863abe79f7742e2101"} err="failed to get container status \"091186c4a45bd814266ac30e567725721c206e48735026863abe79f7742e2101\": rpc error: code = NotFound desc = could not find container \"091186c4a45bd814266ac30e567725721c206e48735026863abe79f7742e2101\": container with ID starting with 091186c4a45bd814266ac30e567725721c206e48735026863abe79f7742e2101 not found: ID does not exist" Jul 28 13:49:49.344995 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.344932 2550 scope.go:117] "RemoveContainer" containerID="b0f3145a81cc40b101d3797e3ccb2968894ee11d801f5d32fc338d47d802f1fc" Jul 28 13:49:49.345130 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.345110 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0f3145a81cc40b101d3797e3ccb2968894ee11d801f5d32fc338d47d802f1fc"} err="failed to get container status \"b0f3145a81cc40b101d3797e3ccb2968894ee11d801f5d32fc338d47d802f1fc\": rpc error: code = NotFound desc = could not find container \"b0f3145a81cc40b101d3797e3ccb2968894ee11d801f5d32fc338d47d802f1fc\": container with ID starting with b0f3145a81cc40b101d3797e3ccb2968894ee11d801f5d32fc338d47d802f1fc not found: ID does not exist" Jul 28 13:49:49.345188 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.345130 2550 scope.go:117] "RemoveContainer" containerID="727c52f4873cc6b2b374041bd49ccdb19cf16ea52fbacd85505e857623c03963" Jul 28 13:49:49.345377 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.345347 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"727c52f4873cc6b2b374041bd49ccdb19cf16ea52fbacd85505e857623c03963"} err="failed to get container status \"727c52f4873cc6b2b374041bd49ccdb19cf16ea52fbacd85505e857623c03963\": rpc error: code = NotFound desc = could not find container \"727c52f4873cc6b2b374041bd49ccdb19cf16ea52fbacd85505e857623c03963\": container with ID starting with 727c52f4873cc6b2b374041bd49ccdb19cf16ea52fbacd85505e857623c03963 not found: ID does not exist" Jul 28 13:49:49.345750 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.345732 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 13:49:49.346083 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.346069 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" containerName="thanos-sidecar" Jul 28 13:49:49.346156 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.346087 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" containerName="thanos-sidecar" Jul 28 13:49:49.346156 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.346101 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" containerName="kube-rbac-proxy-web" Jul 28 13:49:49.346156 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.346111 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" containerName="kube-rbac-proxy-web" Jul 28 13:49:49.346156 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.346127 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" containerName="prometheus" Jul 28 13:49:49.346156 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.346136 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" containerName="prometheus" Jul 28 13:49:49.346156 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.346147 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" containerName="init-config-reloader" Jul 28 13:49:49.346156 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.346156 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" containerName="init-config-reloader" Jul 28 13:49:49.346513 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.346175 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" containerName="config-reloader" Jul 28 13:49:49.346513 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.346184 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" containerName="config-reloader" Jul 28 13:49:49.346513 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.346202 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" containerName="kube-rbac-proxy" Jul 28 13:49:49.346513 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.346210 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" containerName="kube-rbac-proxy" Jul 28 13:49:49.346513 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.346226 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" containerName="kube-rbac-proxy-thanos" Jul 28 13:49:49.346513 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.346234 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" containerName="kube-rbac-proxy-thanos" Jul 28 13:49:49.346513 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.346313 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" containerName="kube-rbac-proxy-thanos" Jul 28 13:49:49.346513 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.346325 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" containerName="prometheus" Jul 28 13:49:49.346513 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.346335 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" containerName="thanos-sidecar" Jul 28 13:49:49.346513 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.346347 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" containerName="kube-rbac-proxy-web" Jul 28 13:49:49.346513 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.346358 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" containerName="kube-rbac-proxy" Jul 28 13:49:49.346513 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.346369 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" containerName="config-reloader" Jul 28 13:49:49.350867 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.350843 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.353673 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.353649 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 28 13:49:49.353874 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.353846 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 28 13:49:49.353874 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.353869 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 28 13:49:49.354015 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.353891 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 28 13:49:49.354015 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.353964 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-s2wvd\"" Jul 28 13:49:49.354122 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.354093 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 28 13:49:49.354320 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.354300 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 28 13:49:49.354431 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.354302 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 28 13:49:49.354431 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.354427 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 28 13:49:49.354518 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.354480 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 28 13:49:49.354567 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.354521 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 28 13:49:49.354619 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.354571 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 28 13:49:49.354847 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.354827 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-e7pgtvnm40u92\"" Jul 28 13:49:49.355480 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.355457 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 28 13:49:49.358198 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.357123 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 28 13:49:49.360281 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.360259 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 28 13:49:49.368216 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.368186 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 13:49:49.451705 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.451609 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b079ff8a-455d-4e5b-acc6-d3cfb8995523-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.451705 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.451651 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b079ff8a-455d-4e5b-acc6-d3cfb8995523-config-out\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.451705 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.451670 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b079ff8a-455d-4e5b-acc6-d3cfb8995523-web-config\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.451918 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.451736 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b079ff8a-455d-4e5b-acc6-d3cfb8995523-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.451918 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.451776 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b079ff8a-455d-4e5b-acc6-d3cfb8995523-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.451918 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.451796 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b079ff8a-455d-4e5b-acc6-d3cfb8995523-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.451918 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.451813 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b079ff8a-455d-4e5b-acc6-d3cfb8995523-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.451918 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.451828 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b079ff8a-455d-4e5b-acc6-d3cfb8995523-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.451918 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.451909 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b079ff8a-455d-4e5b-acc6-d3cfb8995523-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.452099 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.451943 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b079ff8a-455d-4e5b-acc6-d3cfb8995523-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.452099 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.451960 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b079ff8a-455d-4e5b-acc6-d3cfb8995523-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.452099 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.451978 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b079ff8a-455d-4e5b-acc6-d3cfb8995523-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.452099 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.452000 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b079ff8a-455d-4e5b-acc6-d3cfb8995523-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.452099 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.452017 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b079ff8a-455d-4e5b-acc6-d3cfb8995523-config\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.452099 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.452037 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b079ff8a-455d-4e5b-acc6-d3cfb8995523-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.452099 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.452057 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b079ff8a-455d-4e5b-acc6-d3cfb8995523-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.452099 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.452073 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b079ff8a-455d-4e5b-acc6-d3cfb8995523-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.452099 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.452095 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b079ff8a-455d-4e5b-acc6-d3cfb8995523-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.452430 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.452129 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b079ff8a-455d-4e5b-acc6-d3cfb8995523-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.452430 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.452155 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m29zk\" (UniqueName: \"kubernetes.io/projected/b079ff8a-455d-4e5b-acc6-d3cfb8995523-kube-api-access-m29zk\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.553376 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.553337 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b079ff8a-455d-4e5b-acc6-d3cfb8995523-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.553555 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.553386 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b079ff8a-455d-4e5b-acc6-d3cfb8995523-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.553555 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.553433 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b079ff8a-455d-4e5b-acc6-d3cfb8995523-config\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.553555 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.553458 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b079ff8a-455d-4e5b-acc6-d3cfb8995523-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.553555 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.553491 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b079ff8a-455d-4e5b-acc6-d3cfb8995523-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.553555 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.553512 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b079ff8a-455d-4e5b-acc6-d3cfb8995523-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.553773 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.553598 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b079ff8a-455d-4e5b-acc6-d3cfb8995523-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.555878 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.554285 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b079ff8a-455d-4e5b-acc6-d3cfb8995523-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.555878 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.554320 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b079ff8a-455d-4e5b-acc6-d3cfb8995523-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.555878 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.554341 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m29zk\" (UniqueName: \"kubernetes.io/projected/b079ff8a-455d-4e5b-acc6-d3cfb8995523-kube-api-access-m29zk\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.555878 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.554422 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b079ff8a-455d-4e5b-acc6-d3cfb8995523-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.555878 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.554636 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b079ff8a-455d-4e5b-acc6-d3cfb8995523-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.555878 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.554687 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b079ff8a-455d-4e5b-acc6-d3cfb8995523-config-out\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.555878 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.554855 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b079ff8a-455d-4e5b-acc6-d3cfb8995523-web-config\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.558644 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.556861 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/b079ff8a-455d-4e5b-acc6-d3cfb8995523-config\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.558644 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.556954 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b079ff8a-455d-4e5b-acc6-d3cfb8995523-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.558644 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.557292 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b079ff8a-455d-4e5b-acc6-d3cfb8995523-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.558644 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.557413 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b079ff8a-455d-4e5b-acc6-d3cfb8995523-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.558644 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.557512 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b079ff8a-455d-4e5b-acc6-d3cfb8995523-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.558644 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.557601 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b079ff8a-455d-4e5b-acc6-d3cfb8995523-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.558644 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.557698 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b079ff8a-455d-4e5b-acc6-d3cfb8995523-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.558644 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.557830 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b079ff8a-455d-4e5b-acc6-d3cfb8995523-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.558644 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.557944 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b079ff8a-455d-4e5b-acc6-d3cfb8995523-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.558644 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.558002 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b079ff8a-455d-4e5b-acc6-d3cfb8995523-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.558644 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.558515 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b079ff8a-455d-4e5b-acc6-d3cfb8995523-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.559272 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.558544 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b079ff8a-455d-4e5b-acc6-d3cfb8995523-web-config\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.559272 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.559172 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b079ff8a-455d-4e5b-acc6-d3cfb8995523-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.559953 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.559720 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b079ff8a-455d-4e5b-acc6-d3cfb8995523-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.561356 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.561318 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b079ff8a-455d-4e5b-acc6-d3cfb8995523-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.561666 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.561645 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b079ff8a-455d-4e5b-acc6-d3cfb8995523-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.561744 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.561699 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b079ff8a-455d-4e5b-acc6-d3cfb8995523-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.563195 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.562041 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b079ff8a-455d-4e5b-acc6-d3cfb8995523-config-out\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.563195 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.562724 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b079ff8a-455d-4e5b-acc6-d3cfb8995523-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.563195 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.562755 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b079ff8a-455d-4e5b-acc6-d3cfb8995523-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.563409 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.563223 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b079ff8a-455d-4e5b-acc6-d3cfb8995523-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.563559 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.563527 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b079ff8a-455d-4e5b-acc6-d3cfb8995523-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.564454 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.564430 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b079ff8a-455d-4e5b-acc6-d3cfb8995523-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.564555 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.564470 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b079ff8a-455d-4e5b-acc6-d3cfb8995523-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.565698 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.565679 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b079ff8a-455d-4e5b-acc6-d3cfb8995523-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.566820 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.566799 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m29zk\" (UniqueName: \"kubernetes.io/projected/b079ff8a-455d-4e5b-acc6-d3cfb8995523-kube-api-access-m29zk\") pod \"prometheus-k8s-0\" (UID: \"b079ff8a-455d-4e5b-acc6-d3cfb8995523\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.661862 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.661828 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:49:49.800926 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:49.800901 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 13:49:49.802840 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:49:49.802817 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb079ff8a_455d_4e5b_acc6_d3cfb8995523.slice/crio-4e833b24d20a29836d076a363a72301485099a88b0b860e378e8ea791108bd0c WatchSource:0}: Error finding container 4e833b24d20a29836d076a363a72301485099a88b0b860e378e8ea791108bd0c: Status 404 returned error can't find the container with id 4e833b24d20a29836d076a363a72301485099a88b0b860e378e8ea791108bd0c Jul 28 13:49:50.291860 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:50.291825 2550 generic.go:356] "Generic (PLEG): container finished" podID="b079ff8a-455d-4e5b-acc6-d3cfb8995523" containerID="c46871572d1b3ab3ffc223957219aaeea47d77e01b8be7411ffb910a8d4ec792" exitCode=0 Jul 28 13:49:50.292277 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:50.291914 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b079ff8a-455d-4e5b-acc6-d3cfb8995523","Type":"ContainerDied","Data":"c46871572d1b3ab3ffc223957219aaeea47d77e01b8be7411ffb910a8d4ec792"} Jul 28 13:49:50.292277 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:50.291983 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b079ff8a-455d-4e5b-acc6-d3cfb8995523","Type":"ContainerStarted","Data":"4e833b24d20a29836d076a363a72301485099a88b0b860e378e8ea791108bd0c"} Jul 28 13:49:50.480190 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:50.480148 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d" path="/var/lib/kubelet/pods/cfd1b45c-a81e-41f9-8a6b-bbf1b7d2106d/volumes" Jul 28 13:49:51.299015 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:51.298979 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b079ff8a-455d-4e5b-acc6-d3cfb8995523","Type":"ContainerStarted","Data":"5748d6586a888991f54eefce21a38931b8b771c3cf31c20feadb79f622beeafb"} Jul 28 13:49:51.299508 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:51.299055 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b079ff8a-455d-4e5b-acc6-d3cfb8995523","Type":"ContainerStarted","Data":"0a9c7e90df9791af73cee7cdb656ef1ee7a3a1aa8dfddbfb59cf3eed4a7eeea6"} Jul 28 13:49:51.299508 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:51.299102 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b079ff8a-455d-4e5b-acc6-d3cfb8995523","Type":"ContainerStarted","Data":"858ed623857f067b17587c430717454ce21f2705c39279f66a50c650bf86a882"} Jul 28 13:49:51.299508 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:51.299145 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b079ff8a-455d-4e5b-acc6-d3cfb8995523","Type":"ContainerStarted","Data":"801b20f69ac8624601651b5559fdda775158b1b8724320bb60a7c062cec9cb67"} Jul 28 13:49:51.299508 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:51.299188 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b079ff8a-455d-4e5b-acc6-d3cfb8995523","Type":"ContainerStarted","Data":"870d9a705a37ed78d033c82ed2bb70af2112f6ab78716f1f1ed4f76af3ff7379"} Jul 28 13:49:51.299508 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:51.299224 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b079ff8a-455d-4e5b-acc6-d3cfb8995523","Type":"ContainerStarted","Data":"1fe85c5058f75fced7d97305e5baab218af474db667adf110dc6940486a6f20e"} Jul 28 13:49:51.333135 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:51.333076 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.311010553 podStartE2EDuration="2.333053846s" totalImagesPullingTime="0s" totalInitContainerRuntime="22.043293ms" isStatefulPod=false podCreationTimestamp="2026-07-28 13:49:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:49:51.331030112 +0000 UTC m=+145.466227005" watchObservedRunningTime="2026-07-28 13:49:51.333053846 +0000 UTC m=+145.468250739" Jul 28 13:49:54.662595 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:49:54.662545 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:50:49.662904 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:50:49.662867 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:50:49.678547 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:50:49.678521 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:50:50.548004 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:50:50.547978 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 13:51:30.464651 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:51:30.464615 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-2dxx7"] Jul 28 13:51:30.467145 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:51:30.467124 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-2dxx7" Jul 28 13:51:30.469621 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:51:30.469595 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 28 13:51:30.469730 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:51:30.469626 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-t4gcg\"" Jul 28 13:51:30.469730 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:51:30.469632 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 28 13:51:30.477641 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:51:30.477618 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-2dxx7"] Jul 28 13:51:30.559768 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:51:30.559736 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9a00a9df-09f4-4f0d-b025-f88693a5ec6a-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-2dxx7\" (UID: \"9a00a9df-09f4-4f0d-b025-f88693a5ec6a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2dxx7" Jul 28 13:51:30.559926 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:51:30.559785 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9w6cj\" (UniqueName: \"kubernetes.io/projected/9a00a9df-09f4-4f0d-b025-f88693a5ec6a-kube-api-access-9w6cj\") pod \"cert-manager-webhook-6486ffd997-2dxx7\" (UID: \"9a00a9df-09f4-4f0d-b025-f88693a5ec6a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2dxx7" Jul 28 13:51:30.661047 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:51:30.661014 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9a00a9df-09f4-4f0d-b025-f88693a5ec6a-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-2dxx7\" (UID: \"9a00a9df-09f4-4f0d-b025-f88693a5ec6a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2dxx7" Jul 28 13:51:30.661214 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:51:30.661064 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w6cj\" (UniqueName: \"kubernetes.io/projected/9a00a9df-09f4-4f0d-b025-f88693a5ec6a-kube-api-access-9w6cj\") pod \"cert-manager-webhook-6486ffd997-2dxx7\" (UID: \"9a00a9df-09f4-4f0d-b025-f88693a5ec6a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2dxx7" Jul 28 13:51:30.671238 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:51:30.671214 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9w6cj\" (UniqueName: \"kubernetes.io/projected/9a00a9df-09f4-4f0d-b025-f88693a5ec6a-kube-api-access-9w6cj\") pod \"cert-manager-webhook-6486ffd997-2dxx7\" (UID: \"9a00a9df-09f4-4f0d-b025-f88693a5ec6a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2dxx7" Jul 28 13:51:30.671353 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:51:30.671214 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9a00a9df-09f4-4f0d-b025-f88693a5ec6a-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-2dxx7\" (UID: \"9a00a9df-09f4-4f0d-b025-f88693a5ec6a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2dxx7" Jul 28 13:51:30.782757 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:51:30.782673 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-2dxx7" Jul 28 13:51:30.911779 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:51:30.911757 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-2dxx7"] Jul 28 13:51:30.914047 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:51:30.914023 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a00a9df_09f4_4f0d_b025_f88693a5ec6a.slice/crio-fb14203c36598233593678adb5816c5e3b40c1be4fa56278777a402f83e4766e WatchSource:0}: Error finding container fb14203c36598233593678adb5816c5e3b40c1be4fa56278777a402f83e4766e: Status 404 returned error can't find the container with id fb14203c36598233593678adb5816c5e3b40c1be4fa56278777a402f83e4766e Jul 28 13:51:31.693815 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:51:31.693781 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-2dxx7" event={"ID":"9a00a9df-09f4-4f0d-b025-f88693a5ec6a","Type":"ContainerStarted","Data":"fb14203c36598233593678adb5816c5e3b40c1be4fa56278777a402f83e4766e"} Jul 28 13:51:32.052979 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:51:32.052854 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-cqfx5"] Jul 28 13:51:32.055742 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:51:32.055713 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-cqfx5" Jul 28 13:51:32.058731 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:51:32.058519 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-vld8j\"" Jul 28 13:51:32.069887 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:51:32.069863 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-cqfx5"] Jul 28 13:51:32.173842 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:51:32.173697 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a286f30c-fcb1-4ba9-8039-6799227da72e-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-cqfx5\" (UID: \"a286f30c-fcb1-4ba9-8039-6799227da72e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cqfx5" Jul 28 13:51:32.173842 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:51:32.173747 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-495bt\" (UniqueName: \"kubernetes.io/projected/a286f30c-fcb1-4ba9-8039-6799227da72e-kube-api-access-495bt\") pod \"cert-manager-cainjector-559cdd8c87-cqfx5\" (UID: \"a286f30c-fcb1-4ba9-8039-6799227da72e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cqfx5" Jul 28 13:51:32.275352 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:51:32.275297 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a286f30c-fcb1-4ba9-8039-6799227da72e-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-cqfx5\" (UID: \"a286f30c-fcb1-4ba9-8039-6799227da72e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cqfx5" Jul 28 13:51:32.275352 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:51:32.275347 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-495bt\" (UniqueName: \"kubernetes.io/projected/a286f30c-fcb1-4ba9-8039-6799227da72e-kube-api-access-495bt\") pod \"cert-manager-cainjector-559cdd8c87-cqfx5\" (UID: \"a286f30c-fcb1-4ba9-8039-6799227da72e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cqfx5" Jul 28 13:51:32.286524 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:51:32.286074 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a286f30c-fcb1-4ba9-8039-6799227da72e-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-cqfx5\" (UID: \"a286f30c-fcb1-4ba9-8039-6799227da72e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cqfx5" Jul 28 13:51:32.286524 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:51:32.286173 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-495bt\" (UniqueName: \"kubernetes.io/projected/a286f30c-fcb1-4ba9-8039-6799227da72e-kube-api-access-495bt\") pod \"cert-manager-cainjector-559cdd8c87-cqfx5\" (UID: \"a286f30c-fcb1-4ba9-8039-6799227da72e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cqfx5" Jul 28 13:51:32.371213 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:51:32.370881 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-cqfx5" Jul 28 13:51:32.540627 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:51:32.540554 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-cqfx5"] Jul 28 13:51:32.543746 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:51:32.543717 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda286f30c_fcb1_4ba9_8039_6799227da72e.slice/crio-c0b000b57ec3a8a5ef351f2e7248699259c3f3bbd544fda765eb73c66fc28cb0 WatchSource:0}: Error finding container c0b000b57ec3a8a5ef351f2e7248699259c3f3bbd544fda765eb73c66fc28cb0: Status 404 returned error can't find the container with id c0b000b57ec3a8a5ef351f2e7248699259c3f3bbd544fda765eb73c66fc28cb0 Jul 28 13:51:32.700949 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:51:32.700901 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-cqfx5" event={"ID":"a286f30c-fcb1-4ba9-8039-6799227da72e","Type":"ContainerStarted","Data":"c0b000b57ec3a8a5ef351f2e7248699259c3f3bbd544fda765eb73c66fc28cb0"} Jul 28 13:51:34.712452 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:51:34.712415 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-2dxx7" event={"ID":"9a00a9df-09f4-4f0d-b025-f88693a5ec6a","Type":"ContainerStarted","Data":"f844538652d42ee8dcb8b05d6686a9eb1c22bba4a82b5932d89fc608b53552fb"} Jul 28 13:51:34.712933 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:51:34.712485 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-2dxx7" Jul 28 13:51:34.714157 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:51:34.714131 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-cqfx5" event={"ID":"a286f30c-fcb1-4ba9-8039-6799227da72e","Type":"ContainerStarted","Data":"94a069c92e7749fb917f3b4689a9d79e0090b941ce4bcbd0be65c28fd0d7d545"} Jul 28 13:51:34.731140 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:51:34.731090 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-2dxx7" podStartSLOduration=1.387780588 podStartE2EDuration="4.731077433s" totalImagesPullingTime="3.343296845s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:51:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:51:34.729890949 +0000 UTC m=+248.865087843" watchObservedRunningTime="2026-07-28 13:51:34.731077433 +0000 UTC m=+248.866274324" Jul 28 13:51:34.746710 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:51:34.746667 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-cqfx5" podStartSLOduration=1.032564303 podStartE2EDuration="2.746651764s" totalImagesPullingTime="1.714087461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:51:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:51:34.745189009 +0000 UTC m=+248.880385904" watchObservedRunningTime="2026-07-28 13:51:34.746651764 +0000 UTC m=+248.881848656" Jul 28 13:51:40.720261 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:51:40.720228 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-2dxx7" Jul 28 13:52:26.302129 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:52:26.302103 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4nb4j_f498bb4f-ae84-4a84-ba7d-189d4a713911/ovn-acl-logging/0.log" Jul 28 13:52:26.305919 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:52:26.305885 2550 kubelet.go:1648] "Image garbage collection succeeded" Jul 28 13:52:26.311236 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:52:26.311215 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4nb4j_f498bb4f-ae84-4a84-ba7d-189d4a713911/ovn-acl-logging/0.log" Jul 28 13:54:28.667939 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:54:28.667897 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-trainjob-workflow-node-0-0-5vpcz"] Jul 28 13:54:28.671452 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:54:28.671436 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-trainjob-workflow-node-0-0-5vpcz" Jul 28 13:54:28.674089 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:54:28.674063 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 28 13:54:28.674089 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:54:28.674065 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 28 13:54:28.674271 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:54:28.674139 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-xbrt5\"" Jul 28 13:54:28.680136 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:54:28.680112 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-trainjob-workflow-node-0-0-5vpcz"] Jul 28 13:54:28.805407 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:54:28.805350 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-252sv\" (UniqueName: \"kubernetes.io/projected/5c5910bb-f0f3-4b57-9e0d-5dfcb66f5a69-kube-api-access-252sv\") pod \"test-trainjob-workflow-node-0-0-5vpcz\" (UID: \"5c5910bb-f0f3-4b57-9e0d-5dfcb66f5a69\") " pod="opendatahub/test-trainjob-workflow-node-0-0-5vpcz" Jul 28 13:54:28.906744 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:54:28.906692 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-252sv\" (UniqueName: \"kubernetes.io/projected/5c5910bb-f0f3-4b57-9e0d-5dfcb66f5a69-kube-api-access-252sv\") pod \"test-trainjob-workflow-node-0-0-5vpcz\" (UID: \"5c5910bb-f0f3-4b57-9e0d-5dfcb66f5a69\") " pod="opendatahub/test-trainjob-workflow-node-0-0-5vpcz" Jul 28 13:54:28.916673 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:54:28.916641 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-252sv\" (UniqueName: \"kubernetes.io/projected/5c5910bb-f0f3-4b57-9e0d-5dfcb66f5a69-kube-api-access-252sv\") pod \"test-trainjob-workflow-node-0-0-5vpcz\" (UID: \"5c5910bb-f0f3-4b57-9e0d-5dfcb66f5a69\") " pod="opendatahub/test-trainjob-workflow-node-0-0-5vpcz" Jul 28 13:54:28.981226 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:54:28.981154 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-trainjob-workflow-node-0-0-5vpcz" Jul 28 13:54:29.114829 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:54:29.114786 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-trainjob-workflow-node-0-0-5vpcz"] Jul 28 13:54:29.117191 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:54:29.117163 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5c5910bb_f0f3_4b57_9e0d_5dfcb66f5a69.slice/crio-c54bd09697d457f4a18659a87e8f58b0af1c4046b44d4ced60abda442cca80ad WatchSource:0}: Error finding container c54bd09697d457f4a18659a87e8f58b0af1c4046b44d4ced60abda442cca80ad: Status 404 returned error can't find the container with id c54bd09697d457f4a18659a87e8f58b0af1c4046b44d4ced60abda442cca80ad Jul 28 13:54:29.119343 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:54:29.119327 2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 13:54:29.431131 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:54:29.431095 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-trainjob-workflow-node-0-0-5vpcz" event={"ID":"5c5910bb-f0f3-4b57-9e0d-5dfcb66f5a69","Type":"ContainerStarted","Data":"c54bd09697d457f4a18659a87e8f58b0af1c4046b44d4ced60abda442cca80ad"} Jul 28 13:55:36.794640 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:55:36.794601 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-trainjob-workflow-node-0-0-5vpcz" event={"ID":"5c5910bb-f0f3-4b57-9e0d-5dfcb66f5a69","Type":"ContainerStarted","Data":"c543d426eb52ca6be13ef074af760ea7b8d5f756df6c0002e6e5d35e92576305"} Jul 28 13:55:36.815817 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:55:36.815751 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-trainjob-workflow-node-0-0-5vpcz" podStartSLOduration=1.856660217 podStartE2EDuration="1m8.815731434s" totalImagesPullingTime="1m6.959071217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:54:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:55:36.814066246 +0000 UTC m=+490.949263140" watchObservedRunningTime="2026-07-28 13:55:36.815731434 +0000 UTC m=+490.950928329" Jul 28 13:55:39.808136 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:55:39.808103 2550 generic.go:356] "Generic (PLEG): container finished" podID="5c5910bb-f0f3-4b57-9e0d-5dfcb66f5a69" containerID="c543d426eb52ca6be13ef074af760ea7b8d5f756df6c0002e6e5d35e92576305" exitCode=0 Jul 28 13:55:39.808545 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:55:39.808152 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-trainjob-workflow-node-0-0-5vpcz" event={"ID":"5c5910bb-f0f3-4b57-9e0d-5dfcb66f5a69","Type":"ContainerDied","Data":"c543d426eb52ca6be13ef074af760ea7b8d5f756df6c0002e6e5d35e92576305"} Jul 28 13:55:40.971554 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:55:40.971522 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-trainjob-workflow-node-0-0-5vpcz" Jul 28 13:55:41.102204 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:55:41.102125 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-252sv\" (UniqueName: \"kubernetes.io/projected/5c5910bb-f0f3-4b57-9e0d-5dfcb66f5a69-kube-api-access-252sv\") pod \"5c5910bb-f0f3-4b57-9e0d-5dfcb66f5a69\" (UID: \"5c5910bb-f0f3-4b57-9e0d-5dfcb66f5a69\") " Jul 28 13:55:41.104271 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:55:41.104239 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c5910bb-f0f3-4b57-9e0d-5dfcb66f5a69-kube-api-access-252sv" (OuterVolumeSpecName: "kube-api-access-252sv") pod "5c5910bb-f0f3-4b57-9e0d-5dfcb66f5a69" (UID: "5c5910bb-f0f3-4b57-9e0d-5dfcb66f5a69"). InnerVolumeSpecName "kube-api-access-252sv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 13:55:41.202950 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:55:41.202914 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-252sv\" (UniqueName: \"kubernetes.io/projected/5c5910bb-f0f3-4b57-9e0d-5dfcb66f5a69-kube-api-access-252sv\") on node \"ip-10-0-141-63.ec2.internal\" DevicePath \"\"" Jul 28 13:55:41.817262 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:55:41.817231 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-trainjob-workflow-node-0-0-5vpcz" Jul 28 13:55:41.817448 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:55:41.817264 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-trainjob-workflow-node-0-0-5vpcz" event={"ID":"5c5910bb-f0f3-4b57-9e0d-5dfcb66f5a69","Type":"ContainerDied","Data":"c54bd09697d457f4a18659a87e8f58b0af1c4046b44d4ced60abda442cca80ad"} Jul 28 13:55:41.817448 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:55:41.817298 2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c54bd09697d457f4a18659a87e8f58b0af1c4046b44d4ced60abda442cca80ad" Jul 28 13:55:42.045622 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:55:42.045586 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-trainjob-workflow-node-0-0-5vpcz"] Jul 28 13:55:42.048929 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:55:42.048903 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-trainjob-workflow-node-0-0-5vpcz"] Jul 28 13:55:42.477491 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:55:42.477463 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c5910bb-f0f3-4b57-9e0d-5dfcb66f5a69" path="/var/lib/kubelet/pods/5c5910bb-f0f3-4b57-9e0d-5dfcb66f5a69/volumes" Jul 28 13:57:26.332031 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:57:26.331999 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4nb4j_f498bb4f-ae84-4a84-ba7d-189d4a713911/ovn-acl-logging/0.log" Jul 28 13:57:26.337841 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:57:26.337818 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4nb4j_f498bb4f-ae84-4a84-ba7d-189d4a713911/ovn-acl-logging/0.log" Jul 28 13:57:43.077706 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:57:43.077670 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-csjn8/must-gather-swmxl"] Jul 28 13:57:43.078137 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:57:43.078012 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c5910bb-f0f3-4b57-9e0d-5dfcb66f5a69" containerName="node" Jul 28 13:57:43.078137 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:57:43.078022 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c5910bb-f0f3-4b57-9e0d-5dfcb66f5a69" containerName="node" Jul 28 13:57:43.078137 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:57:43.078081 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c5910bb-f0f3-4b57-9e0d-5dfcb66f5a69" containerName="node" Jul 28 13:57:43.080204 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:57:43.080187 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-csjn8/must-gather-swmxl" Jul 28 13:57:43.083565 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:57:43.083546 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-csjn8\"/\"kube-root-ca.crt\"" Jul 28 13:57:43.083728 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:57:43.083710 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-csjn8\"/\"openshift-service-ca.crt\"" Jul 28 13:57:43.084764 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:57:43.084741 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-csjn8\"/\"default-dockercfg-t7sp5\"" Jul 28 13:57:43.106117 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:57:43.106092 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-csjn8/must-gather-swmxl"] Jul 28 13:57:43.169070 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:57:43.169031 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v794b\" (UniqueName: \"kubernetes.io/projected/b9a13c80-bc11-4b6b-954e-2bf5e829a6a4-kube-api-access-v794b\") pod \"must-gather-swmxl\" (UID: \"b9a13c80-bc11-4b6b-954e-2bf5e829a6a4\") " pod="openshift-must-gather-csjn8/must-gather-swmxl" Jul 28 13:57:43.169255 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:57:43.169119 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b9a13c80-bc11-4b6b-954e-2bf5e829a6a4-must-gather-output\") pod \"must-gather-swmxl\" (UID: \"b9a13c80-bc11-4b6b-954e-2bf5e829a6a4\") " pod="openshift-must-gather-csjn8/must-gather-swmxl" Jul 28 13:57:43.269581 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:57:43.269532 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b9a13c80-bc11-4b6b-954e-2bf5e829a6a4-must-gather-output\") pod \"must-gather-swmxl\" (UID: \"b9a13c80-bc11-4b6b-954e-2bf5e829a6a4\") " pod="openshift-must-gather-csjn8/must-gather-swmxl" Jul 28 13:57:43.269786 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:57:43.269634 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v794b\" (UniqueName: \"kubernetes.io/projected/b9a13c80-bc11-4b6b-954e-2bf5e829a6a4-kube-api-access-v794b\") pod \"must-gather-swmxl\" (UID: \"b9a13c80-bc11-4b6b-954e-2bf5e829a6a4\") " pod="openshift-must-gather-csjn8/must-gather-swmxl" Jul 28 13:57:43.269971 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:57:43.269947 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b9a13c80-bc11-4b6b-954e-2bf5e829a6a4-must-gather-output\") pod \"must-gather-swmxl\" (UID: \"b9a13c80-bc11-4b6b-954e-2bf5e829a6a4\") " pod="openshift-must-gather-csjn8/must-gather-swmxl" Jul 28 13:57:43.278913 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:57:43.278891 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v794b\" (UniqueName: \"kubernetes.io/projected/b9a13c80-bc11-4b6b-954e-2bf5e829a6a4-kube-api-access-v794b\") pod \"must-gather-swmxl\" (UID: \"b9a13c80-bc11-4b6b-954e-2bf5e829a6a4\") " pod="openshift-must-gather-csjn8/must-gather-swmxl" Jul 28 13:57:43.388526 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:57:43.388446 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-csjn8/must-gather-swmxl" Jul 28 13:57:43.517752 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:57:43.517722 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-csjn8/must-gather-swmxl"] Jul 28 13:57:43.520261 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:57:43.520230 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb9a13c80_bc11_4b6b_954e_2bf5e829a6a4.slice/crio-bed7c7194139751c1ee81bb1e44f0fe2643cdd3b3f77ccf785e1591c2c84378e WatchSource:0}: Error finding container bed7c7194139751c1ee81bb1e44f0fe2643cdd3b3f77ccf785e1591c2c84378e: Status 404 returned error can't find the container with id bed7c7194139751c1ee81bb1e44f0fe2643cdd3b3f77ccf785e1591c2c84378e Jul 28 13:57:44.338744 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:57:44.338494 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-csjn8/must-gather-swmxl" event={"ID":"b9a13c80-bc11-4b6b-954e-2bf5e829a6a4","Type":"ContainerStarted","Data":"bed7c7194139751c1ee81bb1e44f0fe2643cdd3b3f77ccf785e1591c2c84378e"} Jul 28 13:57:50.369882 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:57:50.369838 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-csjn8/must-gather-swmxl" event={"ID":"b9a13c80-bc11-4b6b-954e-2bf5e829a6a4","Type":"ContainerStarted","Data":"aba65e5a61687491a7429ec09ed814e442c3fcce61bec15059786e51fb105c3c"} Jul 28 13:57:50.370406 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:57:50.369891 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-csjn8/must-gather-swmxl" event={"ID":"b9a13c80-bc11-4b6b-954e-2bf5e829a6a4","Type":"ContainerStarted","Data":"4a709e5b92ef1b2f12f66f636739cb39e25f2e99385719af5011eb146b9f8fac"} Jul 28 13:57:50.388412 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:57:50.388351 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-csjn8/must-gather-swmxl" podStartSLOduration=1.235851129 podStartE2EDuration="7.388336205s" totalImagesPullingTime="6.152485076s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 13:57:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:57:50.386996851 +0000 UTC m=+624.522193746" watchObservedRunningTime="2026-07-28 13:57:50.388336205 +0000 UTC m=+624.523533098" Jul 28 13:58:39.610714 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:39.610632 2550 generic.go:356] "Generic (PLEG): container finished" podID="b9a13c80-bc11-4b6b-954e-2bf5e829a6a4" containerID="4a709e5b92ef1b2f12f66f636739cb39e25f2e99385719af5011eb146b9f8fac" exitCode=0 Jul 28 13:58:39.610714 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:39.610677 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-csjn8/must-gather-swmxl" event={"ID":"b9a13c80-bc11-4b6b-954e-2bf5e829a6a4","Type":"ContainerDied","Data":"4a709e5b92ef1b2f12f66f636739cb39e25f2e99385719af5011eb146b9f8fac"} Jul 28 13:58:39.611145 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:39.610963 2550 scope.go:117] "RemoveContainer" containerID="4a709e5b92ef1b2f12f66f636739cb39e25f2e99385719af5011eb146b9f8fac" Jul 28 13:58:40.313248 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:40.313194 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-csjn8_must-gather-swmxl_b9a13c80-bc11-4b6b-954e-2bf5e829a6a4/gather/0.log" Jul 28 13:58:43.788850 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:43.788810 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-89dlx_f5efacaa-1da7-4ed7-8bfa-2e805507fde7/global-pull-secret-syncer/0.log" Jul 28 13:58:44.000222 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:44.000174 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vvwjc_55845350-bd96-44ac-ba5f-700cb9efdf60/konnectivity-agent/0.log" Jul 28 13:58:44.056230 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:44.056154 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-63.ec2.internal_d99d1c9d97e7e4c15b53cfee9d40b63c/haproxy/0.log" Jul 28 13:58:45.852195 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:45.852159 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-csjn8/must-gather-swmxl"] Jul 28 13:58:45.852684 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:45.852347 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-csjn8/must-gather-swmxl" podUID="b9a13c80-bc11-4b6b-954e-2bf5e829a6a4" containerName="copy" containerID="cri-o://aba65e5a61687491a7429ec09ed814e442c3fcce61bec15059786e51fb105c3c" gracePeriod=2 Jul 28 13:58:45.858428 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:45.858386 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-csjn8/must-gather-swmxl"] Jul 28 13:58:46.096765 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:46.096742 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-csjn8_must-gather-swmxl_b9a13c80-bc11-4b6b-954e-2bf5e829a6a4/copy/0.log" Jul 28 13:58:46.099371 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:46.099355 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-csjn8/must-gather-swmxl" Jul 28 13:58:46.101308 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:58:46.101285 2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-swmxl\" is forbidden: User \"system:node:ip-10-0-141-63.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-csjn8\": no relationship found between node 'ip-10-0-141-63.ec2.internal' and this object" podUID="b9a13c80-bc11-4b6b-954e-2bf5e829a6a4" pod="openshift-must-gather-csjn8/must-gather-swmxl" Jul 28 13:58:46.253408 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:46.253363 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v794b\" (UniqueName: \"kubernetes.io/projected/b9a13c80-bc11-4b6b-954e-2bf5e829a6a4-kube-api-access-v794b\") pod \"b9a13c80-bc11-4b6b-954e-2bf5e829a6a4\" (UID: \"b9a13c80-bc11-4b6b-954e-2bf5e829a6a4\") " Jul 28 13:58:46.253605 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:46.253489 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b9a13c80-bc11-4b6b-954e-2bf5e829a6a4-must-gather-output\") pod \"b9a13c80-bc11-4b6b-954e-2bf5e829a6a4\" (UID: \"b9a13c80-bc11-4b6b-954e-2bf5e829a6a4\") " Jul 28 13:58:46.255801 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:46.255775 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9a13c80-bc11-4b6b-954e-2bf5e829a6a4-kube-api-access-v794b" (OuterVolumeSpecName: "kube-api-access-v794b") pod "b9a13c80-bc11-4b6b-954e-2bf5e829a6a4" (UID: "b9a13c80-bc11-4b6b-954e-2bf5e829a6a4"). InnerVolumeSpecName "kube-api-access-v794b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 13:58:46.256299 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:46.256277 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9a13c80-bc11-4b6b-954e-2bf5e829a6a4-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "b9a13c80-bc11-4b6b-954e-2bf5e829a6a4" (UID: "b9a13c80-bc11-4b6b-954e-2bf5e829a6a4"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 13:58:46.354307 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:46.354271 2550 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b9a13c80-bc11-4b6b-954e-2bf5e829a6a4-must-gather-output\") on node \"ip-10-0-141-63.ec2.internal\" DevicePath \"\"" Jul 28 13:58:46.354307 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:46.354302 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v794b\" (UniqueName: \"kubernetes.io/projected/b9a13c80-bc11-4b6b-954e-2bf5e829a6a4-kube-api-access-v794b\") on node \"ip-10-0-141-63.ec2.internal\" DevicePath \"\"" Jul 28 13:58:46.476525 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:58:46.476482 2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-swmxl\" is forbidden: User \"system:node:ip-10-0-141-63.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-csjn8\": no relationship found between node 'ip-10-0-141-63.ec2.internal' and this object" podUID="b9a13c80-bc11-4b6b-954e-2bf5e829a6a4" pod="openshift-must-gather-csjn8/must-gather-swmxl" Jul 28 13:58:46.477868 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:46.477846 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9a13c80-bc11-4b6b-954e-2bf5e829a6a4" path="/var/lib/kubelet/pods/b9a13c80-bc11-4b6b-954e-2bf5e829a6a4/volumes" Jul 28 13:58:46.640797 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:46.640719 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-csjn8_must-gather-swmxl_b9a13c80-bc11-4b6b-954e-2bf5e829a6a4/copy/0.log" Jul 28 13:58:46.641068 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:46.641042 2550 generic.go:356] "Generic (PLEG): container finished" podID="b9a13c80-bc11-4b6b-954e-2bf5e829a6a4" containerID="aba65e5a61687491a7429ec09ed814e442c3fcce61bec15059786e51fb105c3c" exitCode=143 Jul 28 13:58:46.641120 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:46.641092 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-csjn8/must-gather-swmxl" Jul 28 13:58:46.641154 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:46.641143 2550 scope.go:117] "RemoveContainer" containerID="aba65e5a61687491a7429ec09ed814e442c3fcce61bec15059786e51fb105c3c" Jul 28 13:58:46.651317 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:46.651261 2550 scope.go:117] "RemoveContainer" containerID="4a709e5b92ef1b2f12f66f636739cb39e25f2e99385719af5011eb146b9f8fac" Jul 28 13:58:46.664448 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:46.664425 2550 scope.go:117] "RemoveContainer" containerID="aba65e5a61687491a7429ec09ed814e442c3fcce61bec15059786e51fb105c3c" Jul 28 13:58:46.664709 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:58:46.664693 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aba65e5a61687491a7429ec09ed814e442c3fcce61bec15059786e51fb105c3c\": container with ID starting with aba65e5a61687491a7429ec09ed814e442c3fcce61bec15059786e51fb105c3c not found: ID does not exist" containerID="aba65e5a61687491a7429ec09ed814e442c3fcce61bec15059786e51fb105c3c" Jul 28 13:58:46.664773 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:46.664718 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aba65e5a61687491a7429ec09ed814e442c3fcce61bec15059786e51fb105c3c"} err="failed to get container status \"aba65e5a61687491a7429ec09ed814e442c3fcce61bec15059786e51fb105c3c\": rpc error: code = NotFound desc = could not find container \"aba65e5a61687491a7429ec09ed814e442c3fcce61bec15059786e51fb105c3c\": container with ID starting with aba65e5a61687491a7429ec09ed814e442c3fcce61bec15059786e51fb105c3c not found: ID does not exist" Jul 28 13:58:46.664773 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:46.664736 2550 scope.go:117] "RemoveContainer" containerID="4a709e5b92ef1b2f12f66f636739cb39e25f2e99385719af5011eb146b9f8fac" Jul 28 13:58:46.664981 ip-10-0-141-63 kubenswrapper[2550]: E0728 13:58:46.664959 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a709e5b92ef1b2f12f66f636739cb39e25f2e99385719af5011eb146b9f8fac\": container with ID starting with 4a709e5b92ef1b2f12f66f636739cb39e25f2e99385719af5011eb146b9f8fac not found: ID does not exist" containerID="4a709e5b92ef1b2f12f66f636739cb39e25f2e99385719af5011eb146b9f8fac" Jul 28 13:58:46.665020 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:46.664988 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a709e5b92ef1b2f12f66f636739cb39e25f2e99385719af5011eb146b9f8fac"} err="failed to get container status \"4a709e5b92ef1b2f12f66f636739cb39e25f2e99385719af5011eb146b9f8fac\": rpc error: code = NotFound desc = could not find container \"4a709e5b92ef1b2f12f66f636739cb39e25f2e99385719af5011eb146b9f8fac\": container with ID starting with 4a709e5b92ef1b2f12f66f636739cb39e25f2e99385719af5011eb146b9f8fac not found: ID does not exist" Jul 28 13:58:47.046341 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:47.046313 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8b900ed2-0eb3-4945-b487-0db3fb0d2a34/alertmanager/0.log" Jul 28 13:58:47.090147 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:47.090119 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8b900ed2-0eb3-4945-b487-0db3fb0d2a34/config-reloader/0.log" Jul 28 13:58:47.135931 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:47.135904 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8b900ed2-0eb3-4945-b487-0db3fb0d2a34/kube-rbac-proxy-web/0.log" Jul 28 13:58:47.182487 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:47.182453 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8b900ed2-0eb3-4945-b487-0db3fb0d2a34/kube-rbac-proxy/0.log" Jul 28 13:58:47.243902 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:47.243871 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8b900ed2-0eb3-4945-b487-0db3fb0d2a34/kube-rbac-proxy-metric/0.log" Jul 28 13:58:47.292660 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:47.292626 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8b900ed2-0eb3-4945-b487-0db3fb0d2a34/prom-label-proxy/0.log" Jul 28 13:58:47.356267 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:47.356235 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8b900ed2-0eb3-4945-b487-0db3fb0d2a34/init-config-reloader/0.log" Jul 28 13:58:47.450850 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:47.450820 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-5m62f_14a143a8-6917-4413-8d3b-333cb12e6f7b/kube-state-metrics/0.log" Jul 28 13:58:47.480748 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:47.480719 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-5m62f_14a143a8-6917-4413-8d3b-333cb12e6f7b/kube-rbac-proxy-main/0.log" Jul 28 13:58:47.515848 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:47.515825 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-5m62f_14a143a8-6917-4413-8d3b-333cb12e6f7b/kube-rbac-proxy-self/0.log" Jul 28 13:58:47.575622 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:47.575549 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6cfdb4569f-sxnmp_851f584e-e63a-469b-8043-194d66cce086/metrics-server/0.log" Jul 28 13:58:47.635001 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:47.634974 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-bs6f6_fc17a1fb-5683-4ce6-b54d-f164c9ca95fc/monitoring-plugin/0.log" Jul 28 13:58:47.876925 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:47.876851 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-x4rhq_bc5f9125-f382-4871-92f6-fdd66afeafd3/node-exporter/0.log" Jul 28 13:58:47.902244 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:47.902216 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-x4rhq_bc5f9125-f382-4871-92f6-fdd66afeafd3/kube-rbac-proxy/0.log" Jul 28 13:58:47.933135 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:47.933112 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-x4rhq_bc5f9125-f382-4871-92f6-fdd66afeafd3/init-textfile/0.log" Jul 28 13:58:47.966412 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:47.966363 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-cbclx_8daae3b7-94f7-47e0-98dd-567bde1a0038/kube-rbac-proxy-main/0.log" Jul 28 13:58:47.997255 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:47.997223 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-cbclx_8daae3b7-94f7-47e0-98dd-567bde1a0038/kube-rbac-proxy-self/0.log" Jul 28 13:58:48.028645 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:48.028614 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-cbclx_8daae3b7-94f7-47e0-98dd-567bde1a0038/openshift-state-metrics/0.log" Jul 28 13:58:48.078830 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:48.078807 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b079ff8a-455d-4e5b-acc6-d3cfb8995523/prometheus/0.log" Jul 28 13:58:48.110805 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:48.110778 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b079ff8a-455d-4e5b-acc6-d3cfb8995523/config-reloader/0.log" Jul 28 13:58:48.147570 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:48.147492 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b079ff8a-455d-4e5b-acc6-d3cfb8995523/thanos-sidecar/0.log" Jul 28 13:58:48.181844 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:48.181808 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b079ff8a-455d-4e5b-acc6-d3cfb8995523/kube-rbac-proxy-web/0.log" Jul 28 13:58:48.215662 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:48.215635 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b079ff8a-455d-4e5b-acc6-d3cfb8995523/kube-rbac-proxy/0.log" Jul 28 13:58:48.252765 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:48.252739 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b079ff8a-455d-4e5b-acc6-d3cfb8995523/kube-rbac-proxy-thanos/0.log" Jul 28 13:58:48.282596 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:48.282571 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b079ff8a-455d-4e5b-acc6-d3cfb8995523/init-config-reloader/0.log" Jul 28 13:58:48.320265 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:48.320229 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-26vgk_a1fe2993-d45f-4585-8c28-fd5b0780bf31/prometheus-operator/0.log" Jul 28 13:58:48.348004 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:48.347957 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-26vgk_a1fe2993-d45f-4585-8c28-fd5b0780bf31/kube-rbac-proxy/0.log" Jul 28 13:58:48.379554 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:48.379530 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-bdqn6_4c634ef8-d53e-4282-b7b7-699f55ed22a2/prometheus-operator-admission-webhook/0.log" Jul 28 13:58:48.424382 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:48.424358 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-78699f8669-pw29w_f4d8acff-39b5-4e12-9465-cf293804b5d6/telemeter-client/0.log" Jul 28 13:58:48.453201 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:48.453176 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-78699f8669-pw29w_f4d8acff-39b5-4e12-9465-cf293804b5d6/reload/0.log" Jul 28 13:58:48.481595 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:48.481561 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-78699f8669-pw29w_f4d8acff-39b5-4e12-9465-cf293804b5d6/kube-rbac-proxy/0.log" Jul 28 13:58:48.515732 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:48.515708 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-ff6496dd8-k76hx_4b4c841c-be47-457a-96d4-c2bef7851671/thanos-query/0.log" Jul 28 13:58:48.548669 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:48.548643 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-ff6496dd8-k76hx_4b4c841c-be47-457a-96d4-c2bef7851671/kube-rbac-proxy-web/0.log" Jul 28 13:58:48.575058 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:48.575032 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-ff6496dd8-k76hx_4b4c841c-be47-457a-96d4-c2bef7851671/kube-rbac-proxy/0.log" Jul 28 13:58:48.605227 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:48.605202 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-ff6496dd8-k76hx_4b4c841c-be47-457a-96d4-c2bef7851671/prom-label-proxy/0.log" Jul 28 13:58:48.632421 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:48.632377 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-ff6496dd8-k76hx_4b4c841c-be47-457a-96d4-c2bef7851671/kube-rbac-proxy-rules/0.log" Jul 28 13:58:48.663115 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:48.663081 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-ff6496dd8-k76hx_4b4c841c-be47-457a-96d4-c2bef7851671/kube-rbac-proxy-metrics/0.log" Jul 28 13:58:51.016544 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:51.016511 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-9ws9v_30347318-34fb-44a6-a955-9b3774e1f73c/download-server/0.log" Jul 28 13:58:51.863835 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:51.863800 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8f9qv/perf-node-gather-daemonset-n2nnx"] Jul 28 13:58:51.864184 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:51.864172 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9a13c80-bc11-4b6b-954e-2bf5e829a6a4" containerName="copy" Jul 28 13:58:51.864227 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:51.864186 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9a13c80-bc11-4b6b-954e-2bf5e829a6a4" containerName="copy" Jul 28 13:58:51.864227 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:51.864198 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9a13c80-bc11-4b6b-954e-2bf5e829a6a4" containerName="gather" Jul 28 13:58:51.864227 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:51.864204 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9a13c80-bc11-4b6b-954e-2bf5e829a6a4" containerName="gather" Jul 28 13:58:51.864316 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:51.864252 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9a13c80-bc11-4b6b-954e-2bf5e829a6a4" containerName="copy" Jul 28 13:58:51.864316 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:51.864259 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9a13c80-bc11-4b6b-954e-2bf5e829a6a4" containerName="gather" Jul 28 13:58:51.867306 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:51.867284 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8f9qv/perf-node-gather-daemonset-n2nnx" Jul 28 13:58:51.870236 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:51.870214 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8f9qv\"/\"kube-root-ca.crt\"" Jul 28 13:58:51.870370 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:51.870217 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8f9qv\"/\"openshift-service-ca.crt\"" Jul 28 13:58:51.870370 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:51.870220 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8f9qv\"/\"default-dockercfg-9psnd\"" Jul 28 13:58:51.878049 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:51.878028 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8f9qv/perf-node-gather-daemonset-n2nnx"] Jul 28 13:58:51.902282 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:51.902256 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f970ea63-ebdf-47c1-a45d-5e545b187c12-proc\") pod \"perf-node-gather-daemonset-n2nnx\" (UID: \"f970ea63-ebdf-47c1-a45d-5e545b187c12\") " pod="openshift-must-gather-8f9qv/perf-node-gather-daemonset-n2nnx" Jul 28 13:58:51.902411 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:51.902289 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8db6\" (UniqueName: \"kubernetes.io/projected/f970ea63-ebdf-47c1-a45d-5e545b187c12-kube-api-access-f8db6\") pod \"perf-node-gather-daemonset-n2nnx\" (UID: \"f970ea63-ebdf-47c1-a45d-5e545b187c12\") " pod="openshift-must-gather-8f9qv/perf-node-gather-daemonset-n2nnx" Jul 28 13:58:51.902411 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:51.902365 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f970ea63-ebdf-47c1-a45d-5e545b187c12-podres\") pod \"perf-node-gather-daemonset-n2nnx\" (UID: \"f970ea63-ebdf-47c1-a45d-5e545b187c12\") " pod="openshift-must-gather-8f9qv/perf-node-gather-daemonset-n2nnx" Jul 28 13:58:51.902509 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:51.902412 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f970ea63-ebdf-47c1-a45d-5e545b187c12-lib-modules\") pod \"perf-node-gather-daemonset-n2nnx\" (UID: \"f970ea63-ebdf-47c1-a45d-5e545b187c12\") " pod="openshift-must-gather-8f9qv/perf-node-gather-daemonset-n2nnx" Jul 28 13:58:51.902509 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:51.902445 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f970ea63-ebdf-47c1-a45d-5e545b187c12-sys\") pod \"perf-node-gather-daemonset-n2nnx\" (UID: \"f970ea63-ebdf-47c1-a45d-5e545b187c12\") " pod="openshift-must-gather-8f9qv/perf-node-gather-daemonset-n2nnx" Jul 28 13:58:52.003784 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:52.003752 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f970ea63-ebdf-47c1-a45d-5e545b187c12-podres\") pod \"perf-node-gather-daemonset-n2nnx\" (UID: \"f970ea63-ebdf-47c1-a45d-5e545b187c12\") " pod="openshift-must-gather-8f9qv/perf-node-gather-daemonset-n2nnx" Jul 28 13:58:52.003951 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:52.003797 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f970ea63-ebdf-47c1-a45d-5e545b187c12-lib-modules\") pod \"perf-node-gather-daemonset-n2nnx\" (UID: \"f970ea63-ebdf-47c1-a45d-5e545b187c12\") " pod="openshift-must-gather-8f9qv/perf-node-gather-daemonset-n2nnx" Jul 28 13:58:52.003951 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:52.003837 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f970ea63-ebdf-47c1-a45d-5e545b187c12-sys\") pod \"perf-node-gather-daemonset-n2nnx\" (UID: \"f970ea63-ebdf-47c1-a45d-5e545b187c12\") " pod="openshift-must-gather-8f9qv/perf-node-gather-daemonset-n2nnx" Jul 28 13:58:52.003951 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:52.003870 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f970ea63-ebdf-47c1-a45d-5e545b187c12-proc\") pod \"perf-node-gather-daemonset-n2nnx\" (UID: \"f970ea63-ebdf-47c1-a45d-5e545b187c12\") " pod="openshift-must-gather-8f9qv/perf-node-gather-daemonset-n2nnx" Jul 28 13:58:52.003951 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:52.003925 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f970ea63-ebdf-47c1-a45d-5e545b187c12-proc\") pod \"perf-node-gather-daemonset-n2nnx\" (UID: \"f970ea63-ebdf-47c1-a45d-5e545b187c12\") " pod="openshift-must-gather-8f9qv/perf-node-gather-daemonset-n2nnx" Jul 28 13:58:52.003951 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:52.003932 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f970ea63-ebdf-47c1-a45d-5e545b187c12-sys\") pod \"perf-node-gather-daemonset-n2nnx\" (UID: \"f970ea63-ebdf-47c1-a45d-5e545b187c12\") " pod="openshift-must-gather-8f9qv/perf-node-gather-daemonset-n2nnx" Jul 28 13:58:52.004216 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:52.003923 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f970ea63-ebdf-47c1-a45d-5e545b187c12-podres\") pod \"perf-node-gather-daemonset-n2nnx\" (UID: \"f970ea63-ebdf-47c1-a45d-5e545b187c12\") " pod="openshift-must-gather-8f9qv/perf-node-gather-daemonset-n2nnx" Jul 28 13:58:52.004216 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:52.003960 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8db6\" (UniqueName: \"kubernetes.io/projected/f970ea63-ebdf-47c1-a45d-5e545b187c12-kube-api-access-f8db6\") pod \"perf-node-gather-daemonset-n2nnx\" (UID: \"f970ea63-ebdf-47c1-a45d-5e545b187c12\") " pod="openshift-must-gather-8f9qv/perf-node-gather-daemonset-n2nnx" Jul 28 13:58:52.004216 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:52.003967 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f970ea63-ebdf-47c1-a45d-5e545b187c12-lib-modules\") pod \"perf-node-gather-daemonset-n2nnx\" (UID: \"f970ea63-ebdf-47c1-a45d-5e545b187c12\") " pod="openshift-must-gather-8f9qv/perf-node-gather-daemonset-n2nnx" Jul 28 13:58:52.013155 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:52.013131 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8db6\" (UniqueName: \"kubernetes.io/projected/f970ea63-ebdf-47c1-a45d-5e545b187c12-kube-api-access-f8db6\") pod \"perf-node-gather-daemonset-n2nnx\" (UID: \"f970ea63-ebdf-47c1-a45d-5e545b187c12\") " pod="openshift-must-gather-8f9qv/perf-node-gather-daemonset-n2nnx" Jul 28 13:58:52.177471 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:52.177438 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8f9qv/perf-node-gather-daemonset-n2nnx" Jul 28 13:58:52.306932 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:52.306904 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8f9qv/perf-node-gather-daemonset-n2nnx"] Jul 28 13:58:52.308503 ip-10-0-141-63 kubenswrapper[2550]: W0728 13:58:52.308477 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf970ea63_ebdf_47c1_a45d_5e545b187c12.slice/crio-413f77f26cf6ed5badc658e81ea27e982e4cb6e4ac433a4a7719e44a00601cee WatchSource:0}: Error finding container 413f77f26cf6ed5badc658e81ea27e982e4cb6e4ac433a4a7719e44a00601cee: Status 404 returned error can't find the container with id 413f77f26cf6ed5badc658e81ea27e982e4cb6e4ac433a4a7719e44a00601cee Jul 28 13:58:52.614941 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:52.614854 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qb9g2_30b7a4cf-a585-4366-bab1-28e9037be498/dns/0.log" Jul 28 13:58:52.641268 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:52.641241 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qb9g2_30b7a4cf-a585-4366-bab1-28e9037be498/kube-rbac-proxy/0.log" Jul 28 13:58:52.667776 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:52.667738 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8f9qv/perf-node-gather-daemonset-n2nnx" event={"ID":"f970ea63-ebdf-47c1-a45d-5e545b187c12","Type":"ContainerStarted","Data":"9e34d3c366395a78c65e0d05b9cef8e14984c387902253b3e92ac8795f628c18"} Jul 28 13:58:52.667950 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:52.667858 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8f9qv/perf-node-gather-daemonset-n2nnx" Jul 28 13:58:52.667950 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:52.667918 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8f9qv/perf-node-gather-daemonset-n2nnx" event={"ID":"f970ea63-ebdf-47c1-a45d-5e545b187c12","Type":"ContainerStarted","Data":"413f77f26cf6ed5badc658e81ea27e982e4cb6e4ac433a4a7719e44a00601cee"} Jul 28 13:58:52.668191 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:52.668168 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bgkv9_59ba325f-5a4f-442d-9c36-618faba03f71/dns-node-resolver/0.log" Jul 28 13:58:52.684837 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:52.684796 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8f9qv/perf-node-gather-daemonset-n2nnx" podStartSLOduration=1.6847818270000001 podStartE2EDuration="1.684781827s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 13:58:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 13:58:52.683906544 +0000 UTC m=+686.819103441" watchObservedRunningTime="2026-07-28 13:58:52.684781827 +0000 UTC m=+686.819978740" Jul 28 13:58:53.219967 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:53.219936 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qvpwc_2a365673-18ca-4ce3-9895-3e1f3e6bea6d/node-ca/0.log" Jul 28 13:58:54.472423 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:54.472359 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7q4gj_779de240-4570-4d0d-b38b-d5e64c01ab73/serve-healthcheck-canary/0.log" Jul 28 13:58:55.028597 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:55.028574 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qk8tc_404a2af9-c2b5-49c9-ad60-f7bb293afd8a/kube-rbac-proxy/0.log" Jul 28 13:58:55.054646 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:55.054594 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qk8tc_404a2af9-c2b5-49c9-ad60-f7bb293afd8a/exporter/0.log" Jul 28 13:58:55.079972 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:55.079944 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qk8tc_404a2af9-c2b5-49c9-ad60-f7bb293afd8a/extractor/0.log" Jul 28 13:58:58.682713 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:58:58.682679 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8f9qv/perf-node-gather-daemonset-n2nnx" Jul 28 13:59:02.456386 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:59:02.456346 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-6d5r6_937ca4bc-863d-45d1-a83a-c497cba19b2c/kube-multus/0.log" Jul 28 13:59:02.679016 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:59:02.678985 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8br7q_51d01c0c-2588-432a-bce0-03459112ffb9/kube-multus-additional-cni-plugins/0.log" Jul 28 13:59:02.708018 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:59:02.707933 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8br7q_51d01c0c-2588-432a-bce0-03459112ffb9/egress-router-binary-copy/0.log" Jul 28 13:59:02.734292 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:59:02.734264 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8br7q_51d01c0c-2588-432a-bce0-03459112ffb9/cni-plugins/0.log" Jul 28 13:59:02.759921 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:59:02.759896 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8br7q_51d01c0c-2588-432a-bce0-03459112ffb9/bond-cni-plugin/0.log" Jul 28 13:59:02.786239 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:59:02.786210 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8br7q_51d01c0c-2588-432a-bce0-03459112ffb9/routeoverride-cni/0.log" Jul 28 13:59:02.812486 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:59:02.812425 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8br7q_51d01c0c-2588-432a-bce0-03459112ffb9/whereabouts-cni-bincopy/0.log" Jul 28 13:59:02.838780 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:59:02.838752 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8br7q_51d01c0c-2588-432a-bce0-03459112ffb9/whereabouts-cni/0.log" Jul 28 13:59:03.151026 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:59:03.150996 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-khglj_7b621445-7c29-4e1f-a55e-bc7161202598/network-metrics-daemon/0.log" Jul 28 13:59:03.177537 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:59:03.177508 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-khglj_7b621445-7c29-4e1f-a55e-bc7161202598/kube-rbac-proxy/0.log" Jul 28 13:59:04.073571 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:59:04.073539 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4nb4j_f498bb4f-ae84-4a84-ba7d-189d4a713911/ovn-controller/0.log" Jul 28 13:59:04.096478 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:59:04.096454 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4nb4j_f498bb4f-ae84-4a84-ba7d-189d4a713911/ovn-acl-logging/1.log" Jul 28 13:59:04.096646 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:59:04.096559 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4nb4j_f498bb4f-ae84-4a84-ba7d-189d4a713911/ovn-acl-logging/0.log" Jul 28 13:59:04.120829 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:59:04.120795 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4nb4j_f498bb4f-ae84-4a84-ba7d-189d4a713911/kube-rbac-proxy-node/0.log" Jul 28 13:59:04.148127 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:59:04.148102 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4nb4j_f498bb4f-ae84-4a84-ba7d-189d4a713911/kube-rbac-proxy-ovn-metrics/0.log" Jul 28 13:59:04.174735 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:59:04.174711 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4nb4j_f498bb4f-ae84-4a84-ba7d-189d4a713911/northd/0.log" Jul 28 13:59:04.199617 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:59:04.199589 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4nb4j_f498bb4f-ae84-4a84-ba7d-189d4a713911/nbdb/0.log" Jul 28 13:59:04.225285 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:59:04.225257 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4nb4j_f498bb4f-ae84-4a84-ba7d-189d4a713911/sbdb/0.log" Jul 28 13:59:04.286451 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:59:04.286423 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4nb4j_f498bb4f-ae84-4a84-ba7d-189d4a713911/ovnkube-controller/0.log" Jul 28 13:59:05.864959 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:59:05.864935 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-2z2sg_fbc15680-5aba-470f-b2ee-fcdf596983ae/network-check-target-container/0.log" Jul 28 13:59:06.849446 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:59:06.849381 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-j5v9s_0ebc30d5-9500-40ed-b7f6-0372a013a978/iptables-alerter/0.log" Jul 28 13:59:07.662050 ip-10-0-141-63 kubenswrapper[2550]: I0728 13:59:07.662019 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-v2wxp_266eea55-0f52-46d3-a006-8282aab58caa/tuned/0.log"