Sep 10 03:45:55.732956 ip-10-0-132-74 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 10 03:45:55.732969 ip-10-0-132-74 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 10 03:45:55.732979 ip-10-0-132-74 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 10 03:45:55.733393 ip-10-0-132-74 systemd[1]: Failed to start Kubernetes Kubelet. Sep 10 03:46:04.759950 ip-10-0-132-74 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 10 03:46:04.759964 ip-10-0-132-74 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 5965d598b89e486ca01d3d9162ca7ba7 -- Sep 10 03:47:53.008593 ip-10-0-132-74 systemd[1]: Starting Kubernetes Kubelet... Sep 10 03:47:53.396670 ip-10-0-132-74 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 03:47:53.396670 ip-10-0-132-74 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 10 03:47:53.396670 ip-10-0-132-74 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 03:47:53.396670 ip-10-0-132-74 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 10 03:47:53.396670 ip-10-0-132-74 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 03:47:53.399596 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.399454 2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 10 03:47:53.404432 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404410 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 03:47:53.404432 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404427 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 03:47:53.404432 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404433 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 03:47:53.404432 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404438 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 03:47:53.404432 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404442 2562 feature_gate.go:328] unrecognized feature gate: Example Sep 10 03:47:53.404722 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404446 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 03:47:53.404722 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404450 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 03:47:53.404722 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404454 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 03:47:53.404722 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404479 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 03:47:53.404722 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404485 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 03:47:53.404722 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404489 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 03:47:53.404722 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404492 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 03:47:53.404722 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404496 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 03:47:53.404722 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404500 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 03:47:53.404722 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404503 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 03:47:53.404722 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404510 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 03:47:53.404722 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404516 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 03:47:53.404722 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404521 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 03:47:53.404722 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404525 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 03:47:53.404722 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404529 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 03:47:53.404722 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404534 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 03:47:53.404722 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404538 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 03:47:53.404722 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404542 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 03:47:53.404722 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404547 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 03:47:53.404722 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404551 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 03:47:53.405472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404556 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 03:47:53.405472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404561 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 03:47:53.405472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404565 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 03:47:53.405472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404569 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 03:47:53.405472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404573 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 03:47:53.405472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404580 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 03:47:53.405472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404587 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 03:47:53.405472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404592 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 03:47:53.405472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404596 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 03:47:53.405472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404600 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 03:47:53.405472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404604 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 03:47:53.405472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404608 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 03:47:53.405472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404612 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 03:47:53.405472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404618 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 03:47:53.405472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404622 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 03:47:53.405472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404628 2562 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 03:47:53.405472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404633 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 03:47:53.405472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404638 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 03:47:53.405472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404642 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 03:47:53.405946 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404646 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 03:47:53.405946 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404650 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 03:47:53.405946 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404654 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 03:47:53.405946 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404661 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 03:47:53.405946 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404665 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 03:47:53.405946 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404669 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 03:47:53.405946 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404674 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 03:47:53.405946 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404679 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 03:47:53.405946 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404683 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 03:47:53.405946 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404687 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 03:47:53.405946 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404691 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 03:47:53.405946 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404695 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 03:47:53.405946 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404699 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 03:47:53.405946 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404704 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 03:47:53.405946 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404710 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 03:47:53.405946 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404716 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 03:47:53.405946 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404721 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 03:47:53.405946 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404725 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 03:47:53.405946 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404730 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 03:47:53.406433 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404735 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 03:47:53.406433 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404739 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 03:47:53.406433 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404744 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 03:47:53.406433 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404749 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 03:47:53.406433 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404753 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 03:47:53.406433 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404761 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 03:47:53.406433 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404767 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 03:47:53.406433 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404773 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 03:47:53.406433 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404778 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 03:47:53.406433 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404782 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 03:47:53.406433 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404787 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 03:47:53.406433 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404792 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 03:47:53.406433 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404797 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 03:47:53.406433 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404802 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 03:47:53.406433 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404807 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 03:47:53.406433 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404811 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 03:47:53.406433 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404815 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 03:47:53.406433 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404819 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 03:47:53.406433 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404824 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 03:47:53.406433 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404828 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 03:47:53.406433 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404832 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 03:47:53.406982 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404836 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 03:47:53.406982 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404840 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 03:47:53.406982 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404845 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 03:47:53.406982 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404849 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 03:47:53.406982 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404853 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 03:47:53.406982 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404857 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 03:47:53.406982 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404861 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 03:47:53.406982 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404865 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 03:47:53.406982 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404870 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 03:47:53.406982 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404874 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 03:47:53.406982 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404878 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 03:47:53.406982 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404882 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 03:47:53.406982 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404886 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 03:47:53.406982 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404890 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 03:47:53.406982 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.404895 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 03:47:53.406982 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.405876 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 03:47:53.406982 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.405888 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 03:47:53.406982 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.405893 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 03:47:53.406982 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.405896 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 03:47:53.407467 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.405899 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 03:47:53.407467 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.405904 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 03:47:53.407467 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.405906 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 03:47:53.407467 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.405910 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 03:47:53.407467 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.405913 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 03:47:53.407467 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.405916 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 03:47:53.407467 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.405926 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 03:47:53.407467 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.405929 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 03:47:53.407467 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.405932 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 03:47:53.407467 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.405935 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 03:47:53.407467 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.405937 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 03:47:53.407467 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.405940 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 03:47:53.407467 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.405945 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 03:47:53.407467 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.405948 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 03:47:53.407467 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.405952 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 03:47:53.407467 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.405955 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 03:47:53.407467 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.405958 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 03:47:53.407467 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.405962 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 03:47:53.407921 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.405966 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 03:47:53.407921 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.405970 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 03:47:53.407921 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.405972 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 03:47:53.407921 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.405975 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 03:47:53.407921 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.405978 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 03:47:53.407921 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.405981 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 03:47:53.407921 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.405983 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 03:47:53.407921 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.405987 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 03:47:53.407921 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.405991 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 03:47:53.407921 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.405994 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 03:47:53.407921 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.405997 2562 feature_gate.go:328] unrecognized feature gate: Example Sep 10 03:47:53.407921 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406001 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 03:47:53.407921 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406004 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 03:47:53.407921 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406006 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 03:47:53.407921 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406009 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 03:47:53.407921 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406012 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 03:47:53.407921 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406015 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 03:47:53.407921 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406018 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 03:47:53.407921 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406021 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 03:47:53.407921 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406023 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 03:47:53.408448 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406026 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 03:47:53.408448 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406028 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 03:47:53.408448 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406031 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 03:47:53.408448 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406034 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 03:47:53.408448 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406036 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 03:47:53.408448 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406039 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 03:47:53.408448 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406041 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 03:47:53.408448 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406044 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 03:47:53.408448 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406047 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 03:47:53.408448 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406050 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 03:47:53.408448 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406053 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 03:47:53.408448 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406055 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 03:47:53.408448 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406059 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 03:47:53.408448 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406061 2562 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 03:47:53.408448 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406064 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 03:47:53.408448 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406066 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 03:47:53.408448 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406069 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 03:47:53.408448 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406073 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 03:47:53.408448 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406075 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 03:47:53.408448 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406078 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 03:47:53.408448 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406080 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 03:47:53.408957 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406083 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 03:47:53.408957 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406086 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 03:47:53.408957 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406089 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 03:47:53.408957 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406092 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 03:47:53.408957 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406095 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 03:47:53.408957 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406097 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 03:47:53.408957 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406100 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 03:47:53.408957 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406102 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 03:47:53.408957 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406105 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 03:47:53.408957 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406108 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 03:47:53.408957 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406110 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 03:47:53.408957 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406113 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 03:47:53.408957 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406116 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 03:47:53.408957 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406118 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 03:47:53.408957 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406121 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 03:47:53.408957 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406123 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 03:47:53.408957 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406126 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 03:47:53.408957 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406128 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 03:47:53.408957 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406130 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 03:47:53.408957 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406133 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 03:47:53.409551 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406135 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 03:47:53.409551 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406138 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 03:47:53.409551 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406141 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 03:47:53.409551 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406143 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 03:47:53.409551 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406146 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 03:47:53.409551 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406148 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 03:47:53.409551 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406151 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 03:47:53.409551 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406153 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 03:47:53.409551 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406155 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 03:47:53.409551 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406158 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 03:47:53.409551 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406160 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 03:47:53.409551 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406163 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 03:47:53.409551 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406165 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 03:47:53.409551 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406169 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 03:47:53.409551 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406172 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 03:47:53.409551 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406175 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 03:47:53.409551 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406596 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 03:47:53.409551 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406601 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 03:47:53.409551 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406604 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 03:47:53.410034 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406607 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 03:47:53.410034 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406610 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 03:47:53.410034 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406612 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 03:47:53.410034 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406615 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 03:47:53.410034 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406617 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 03:47:53.410034 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406620 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 03:47:53.410034 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406623 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 03:47:53.410034 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406625 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 03:47:53.410034 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406629 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 03:47:53.410034 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406631 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 03:47:53.410034 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406634 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 03:47:53.410034 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406636 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 03:47:53.410034 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406639 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 03:47:53.410034 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406642 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 03:47:53.410034 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406645 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 03:47:53.410034 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406648 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 03:47:53.410034 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406650 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 03:47:53.410034 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406653 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 03:47:53.410034 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406656 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 03:47:53.410518 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406658 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 03:47:53.410518 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406661 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 03:47:53.410518 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406663 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 03:47:53.410518 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406665 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 03:47:53.410518 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406668 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 03:47:53.410518 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406672 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 03:47:53.410518 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406676 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 03:47:53.410518 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406680 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 03:47:53.410518 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406683 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 03:47:53.410518 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406686 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 03:47:53.410518 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406689 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 03:47:53.410518 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406693 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 03:47:53.410518 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406697 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 03:47:53.410518 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406700 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 03:47:53.410518 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406703 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 03:47:53.410518 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406706 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 03:47:53.410518 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406709 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 03:47:53.410518 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406711 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 03:47:53.410518 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406714 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 03:47:53.410969 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406716 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 03:47:53.410969 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406719 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 03:47:53.410969 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406721 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 03:47:53.410969 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406724 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 03:47:53.410969 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406727 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 03:47:53.410969 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406730 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 03:47:53.410969 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406732 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 03:47:53.410969 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406735 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 03:47:53.410969 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406738 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 03:47:53.410969 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406740 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 03:47:53.410969 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406743 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 03:47:53.410969 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406746 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 03:47:53.410969 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406748 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 03:47:53.410969 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406751 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 03:47:53.410969 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406753 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 03:47:53.410969 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406756 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 03:47:53.410969 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406758 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 03:47:53.410969 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406761 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 03:47:53.410969 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406763 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 03:47:53.410969 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406766 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 03:47:53.411516 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406769 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 03:47:53.411516 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406771 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 03:47:53.411516 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406774 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 03:47:53.411516 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406776 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 03:47:53.411516 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406779 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 03:47:53.411516 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406782 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 03:47:53.411516 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406784 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 03:47:53.411516 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406787 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 03:47:53.411516 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406789 2562 feature_gate.go:328] unrecognized feature gate: Example Sep 10 03:47:53.411516 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406792 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 03:47:53.411516 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406796 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 03:47:53.411516 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406799 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 03:47:53.411516 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406802 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 03:47:53.411516 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406805 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 03:47:53.411516 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406808 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 03:47:53.411516 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406811 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 03:47:53.411516 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406813 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 03:47:53.411516 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406816 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 03:47:53.411516 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406819 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 03:47:53.411516 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406822 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 03:47:53.412048 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406825 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 03:47:53.412048 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406827 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 03:47:53.412048 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406832 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 03:47:53.412048 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406834 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 03:47:53.412048 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406837 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 03:47:53.412048 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406839 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 03:47:53.412048 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406842 2562 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 03:47:53.412048 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406845 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 03:47:53.412048 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406847 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 03:47:53.412048 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406850 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 03:47:53.412048 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406852 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 03:47:53.412048 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406854 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 03:47:53.412048 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406857 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 03:47:53.412048 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406859 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 03:47:53.412048 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406862 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 03:47:53.412048 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406864 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 03:47:53.412048 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406867 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 03:47:53.412048 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.406870 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 03:47:53.412048 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.406951 2562 flags.go:64] FLAG: --address="0.0.0.0" Sep 10 03:47:53.412048 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.406990 2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 10 03:47:53.412048 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.406997 2562 flags.go:64] FLAG: --anonymous-auth="true" Sep 10 03:47:53.412594 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407002 2562 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 10 03:47:53.412594 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407007 2562 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 10 03:47:53.412594 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407010 2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 10 03:47:53.412594 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407015 2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 10 03:47:53.412594 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407019 2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 10 03:47:53.412594 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407023 2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 10 03:47:53.412594 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407026 2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 10 03:47:53.412594 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407030 2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 10 03:47:53.412594 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407036 2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 10 03:47:53.412594 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407039 2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 10 03:47:53.412594 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407042 2562 flags.go:64] FLAG: --cgroup-root="" Sep 10 03:47:53.412594 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407045 2562 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 10 03:47:53.412594 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407048 2562 flags.go:64] FLAG: --client-ca-file="" Sep 10 03:47:53.412594 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407051 2562 flags.go:64] FLAG: --cloud-provider="external" Sep 10 03:47:53.412594 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407054 2562 flags.go:64] FLAG: --cluster-dns="[]" Sep 10 03:47:53.412594 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407059 2562 flags.go:64] FLAG: --cluster-domain="" Sep 10 03:47:53.412594 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407062 2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 10 03:47:53.412594 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407065 2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 10 03:47:53.412594 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407068 2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 10 03:47:53.412594 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407071 2562 flags.go:64] FLAG: --container-log-max-files="5" Sep 10 03:47:53.412594 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407076 2562 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 10 03:47:53.412594 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407079 2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 10 03:47:53.412594 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407082 2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 10 03:47:53.413155 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407085 2562 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 10 03:47:53.413155 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407088 2562 flags.go:64] FLAG: --contention-profiling="false" Sep 10 03:47:53.413155 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407091 2562 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 10 03:47:53.413155 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407094 2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 10 03:47:53.413155 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407097 2562 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 10 03:47:53.413155 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407100 2562 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 10 03:47:53.413155 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407105 2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 10 03:47:53.413155 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407108 2562 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 10 03:47:53.413155 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407111 2562 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 10 03:47:53.413155 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407114 2562 flags.go:64] FLAG: --enable-load-reader="false" Sep 10 03:47:53.413155 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407117 2562 flags.go:64] FLAG: --enable-server="true" Sep 10 03:47:53.413155 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407120 2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 10 03:47:53.413155 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407125 2562 flags.go:64] FLAG: --event-burst="100" Sep 10 03:47:53.413155 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407128 2562 flags.go:64] FLAG: --event-qps="50" Sep 10 03:47:53.413155 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407131 2562 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 10 03:47:53.413155 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407135 2562 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 10 03:47:53.413155 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407138 2562 flags.go:64] FLAG: --eviction-hard="" Sep 10 03:47:53.413155 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407143 2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 10 03:47:53.413155 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407146 2562 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 10 03:47:53.413155 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407149 2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 10 03:47:53.413155 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407152 2562 flags.go:64] FLAG: --eviction-soft="" Sep 10 03:47:53.413155 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407155 2562 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 10 03:47:53.413155 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407158 2562 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 10 03:47:53.413155 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407161 2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 10 03:47:53.413155 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407164 2562 flags.go:64] FLAG: --experimental-mounter-path="" Sep 10 03:47:53.413803 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407168 2562 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 10 03:47:53.413803 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407171 2562 flags.go:64] FLAG: --fail-swap-on="true" Sep 10 03:47:53.413803 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407173 2562 flags.go:64] FLAG: --feature-gates="" Sep 10 03:47:53.413803 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407177 2562 flags.go:64] FLAG: --file-check-frequency="20s" Sep 10 03:47:53.413803 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407181 2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 10 03:47:53.413803 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407184 2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 10 03:47:53.413803 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407187 2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 10 03:47:53.413803 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407190 2562 flags.go:64] FLAG: --healthz-port="10248" Sep 10 03:47:53.413803 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407193 2562 flags.go:64] FLAG: --help="false" Sep 10 03:47:53.413803 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407196 2562 flags.go:64] FLAG: --hostname-override="ip-10-0-132-74.ec2.internal" Sep 10 03:47:53.413803 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407199 2562 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 10 03:47:53.413803 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407202 2562 flags.go:64] FLAG: --http-check-frequency="20s" Sep 10 03:47:53.413803 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407205 2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 10 03:47:53.413803 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407208 2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 10 03:47:53.413803 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407212 2562 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 10 03:47:53.413803 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407215 2562 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 10 03:47:53.413803 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407218 2562 flags.go:64] FLAG: --image-service-endpoint="" Sep 10 03:47:53.413803 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407221 2562 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 10 03:47:53.413803 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407224 2562 flags.go:64] FLAG: --kube-api-burst="100" Sep 10 03:47:53.413803 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407228 2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 10 03:47:53.413803 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407232 2562 flags.go:64] FLAG: --kube-api-qps="50" Sep 10 03:47:53.413803 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407234 2562 flags.go:64] FLAG: --kube-reserved="" Sep 10 03:47:53.413803 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407237 2562 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 10 03:47:53.413803 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407240 2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 10 03:47:53.414475 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407244 2562 flags.go:64] FLAG: --kubelet-cgroups="" Sep 10 03:47:53.414475 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407247 2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 10 03:47:53.414475 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407250 2562 flags.go:64] FLAG: --lock-file="" Sep 10 03:47:53.414475 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407253 2562 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 10 03:47:53.414475 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407256 2562 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 10 03:47:53.414475 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407259 2562 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 10 03:47:53.414475 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407265 2562 flags.go:64] FLAG: --log-json-split-stream="false" Sep 10 03:47:53.414475 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407268 2562 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 10 03:47:53.414475 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407271 2562 flags.go:64] FLAG: --log-text-split-stream="false" Sep 10 03:47:53.414475 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407273 2562 flags.go:64] FLAG: --logging-format="text" Sep 10 03:47:53.414475 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407276 2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 10 03:47:53.414475 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407280 2562 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 10 03:47:53.414475 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407282 2562 flags.go:64] FLAG: --manifest-url="" Sep 10 03:47:53.414475 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407285 2562 flags.go:64] FLAG: --manifest-url-header="" Sep 10 03:47:53.414475 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407305 2562 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 10 03:47:53.414475 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407309 2562 flags.go:64] FLAG: --max-open-files="1000000" Sep 10 03:47:53.414475 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407313 2562 flags.go:64] FLAG: --max-pods="110" Sep 10 03:47:53.414475 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407316 2562 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 10 03:47:53.414475 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407319 2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 10 03:47:53.414475 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407322 2562 flags.go:64] FLAG: --memory-manager-policy="None" Sep 10 03:47:53.414475 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407325 2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 10 03:47:53.414475 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407328 2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 10 03:47:53.414475 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407331 2562 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 10 03:47:53.414475 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407335 2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 10 03:47:53.414475 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407348 2562 flags.go:64] FLAG: --node-status-max-images="50" Sep 10 03:47:53.415089 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407351 2562 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 10 03:47:53.415089 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407354 2562 flags.go:64] FLAG: --oom-score-adj="-999" Sep 10 03:47:53.415089 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407359 2562 flags.go:64] FLAG: --pod-cidr="" Sep 10 03:47:53.415089 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407362 2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 10 03:47:53.415089 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407368 2562 flags.go:64] FLAG: --pod-manifest-path="" Sep 10 03:47:53.415089 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407371 2562 flags.go:64] FLAG: --pod-max-pids="-1" Sep 10 03:47:53.415089 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407374 2562 flags.go:64] FLAG: --pods-per-core="0" Sep 10 03:47:53.415089 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407377 2562 flags.go:64] FLAG: --port="10250" Sep 10 03:47:53.415089 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407382 2562 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 10 03:47:53.415089 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407385 2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-051936d7b98e70bc8" Sep 10 03:47:53.415089 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407388 2562 flags.go:64] FLAG: --qos-reserved="" Sep 10 03:47:53.415089 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407391 2562 flags.go:64] FLAG: --read-only-port="10255" Sep 10 03:47:53.415089 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407394 2562 flags.go:64] FLAG: --register-node="true" Sep 10 03:47:53.415089 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407397 2562 flags.go:64] FLAG: --register-with-taints="" Sep 10 03:47:53.415089 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407401 2562 flags.go:64] FLAG: --registry-burst="10" Sep 10 03:47:53.415089 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407404 2562 flags.go:64] FLAG: --registry-qps="5" Sep 10 03:47:53.415089 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407407 2562 flags.go:64] FLAG: --reserved-cpus="" Sep 10 03:47:53.415089 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407410 2562 flags.go:64] FLAG: --reserved-memory="" Sep 10 03:47:53.415089 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407414 2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 10 03:47:53.415089 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407417 2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 10 03:47:53.415089 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407420 2562 flags.go:64] FLAG: --rotate-certificates="false" Sep 10 03:47:53.415089 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407423 2562 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 10 03:47:53.415089 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407426 2562 flags.go:64] FLAG: --runonce="false" Sep 10 03:47:53.415089 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407429 2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 10 03:47:53.415089 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407432 2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 10 03:47:53.415971 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407435 2562 flags.go:64] FLAG: --seccomp-default="false" Sep 10 03:47:53.415971 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407438 2562 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 10 03:47:53.415971 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407441 2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 10 03:47:53.415971 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407444 2562 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 10 03:47:53.415971 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407448 2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 10 03:47:53.415971 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407451 2562 flags.go:64] FLAG: --storage-driver-password="root" Sep 10 03:47:53.415971 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407454 2562 flags.go:64] FLAG: --storage-driver-secure="false" Sep 10 03:47:53.415971 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407457 2562 flags.go:64] FLAG: --storage-driver-table="stats" Sep 10 03:47:53.415971 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407460 2562 flags.go:64] FLAG: --storage-driver-user="root" Sep 10 03:47:53.415971 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407464 2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 10 03:47:53.415971 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407467 2562 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 10 03:47:53.415971 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407470 2562 flags.go:64] FLAG: --system-cgroups="" Sep 10 03:47:53.415971 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407473 2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 10 03:47:53.415971 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407479 2562 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 10 03:47:53.415971 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407482 2562 flags.go:64] FLAG: --tls-cert-file="" Sep 10 03:47:53.415971 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407488 2562 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 10 03:47:53.415971 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407492 2562 flags.go:64] FLAG: --tls-min-version="" Sep 10 03:47:53.415971 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407495 2562 flags.go:64] FLAG: --tls-private-key-file="" Sep 10 03:47:53.415971 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407498 2562 flags.go:64] FLAG: --topology-manager-policy="none" Sep 10 03:47:53.415971 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407501 2562 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 10 03:47:53.415971 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407504 2562 flags.go:64] FLAG: --topology-manager-scope="container" Sep 10 03:47:53.415971 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407507 2562 flags.go:64] FLAG: --v="2" Sep 10 03:47:53.415971 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407511 2562 flags.go:64] FLAG: --version="false" Sep 10 03:47:53.415971 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407516 2562 flags.go:64] FLAG: --vmodule="" Sep 10 03:47:53.415971 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407520 2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 10 03:47:53.416970 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.407523 2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 10 03:47:53.416970 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407641 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 03:47:53.416970 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407644 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 03:47:53.416970 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407648 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 03:47:53.416970 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407651 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 03:47:53.416970 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407654 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 03:47:53.416970 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407657 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 03:47:53.416970 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407660 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 03:47:53.416970 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407663 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 03:47:53.416970 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407665 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 03:47:53.416970 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407668 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 03:47:53.416970 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407670 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 03:47:53.416970 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407673 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 03:47:53.416970 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407676 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 03:47:53.416970 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407681 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 03:47:53.416970 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407684 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 03:47:53.416970 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407689 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 03:47:53.416970 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407692 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 03:47:53.416970 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407695 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 03:47:53.417472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407698 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 03:47:53.417472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407700 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 03:47:53.417472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407703 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 03:47:53.417472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407707 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 03:47:53.417472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407710 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 03:47:53.417472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407712 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 03:47:53.417472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407715 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 03:47:53.417472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407718 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 03:47:53.417472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407720 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 03:47:53.417472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407723 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 03:47:53.417472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407725 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 03:47:53.417472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407728 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 03:47:53.417472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407731 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 03:47:53.417472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407734 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 03:47:53.417472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407736 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 03:47:53.417472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407739 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 03:47:53.417472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407752 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 03:47:53.417472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407755 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 03:47:53.417472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407757 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 03:47:53.417472 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407760 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 03:47:53.417967 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407763 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 03:47:53.417967 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407766 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 03:47:53.417967 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407768 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 03:47:53.417967 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407771 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 03:47:53.417967 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407773 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 03:47:53.417967 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407776 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 03:47:53.417967 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407779 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 03:47:53.417967 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407781 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 03:47:53.417967 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407784 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 03:47:53.417967 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407788 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 03:47:53.417967 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407791 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 03:47:53.417967 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407794 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 03:47:53.417967 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407797 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 03:47:53.417967 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407799 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 03:47:53.417967 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407802 2562 feature_gate.go:328] unrecognized feature gate: Example Sep 10 03:47:53.417967 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407806 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 03:47:53.417967 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407808 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 03:47:53.417967 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407811 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 03:47:53.417967 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407813 2562 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 03:47:53.417967 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407816 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 03:47:53.418483 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407819 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 03:47:53.418483 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407822 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 03:47:53.418483 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407825 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 03:47:53.418483 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407827 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 03:47:53.418483 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407830 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 03:47:53.418483 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407834 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 03:47:53.418483 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407838 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 03:47:53.418483 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407841 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 03:47:53.418483 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407843 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 03:47:53.418483 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407846 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 03:47:53.418483 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407848 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 03:47:53.418483 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407851 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 03:47:53.418483 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407854 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 03:47:53.418483 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407856 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 03:47:53.418483 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407859 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 03:47:53.418483 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407861 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 03:47:53.418483 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407864 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 03:47:53.418483 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407867 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 03:47:53.418483 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407869 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 03:47:53.418941 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407872 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 03:47:53.418941 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407875 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 03:47:53.418941 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407879 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 03:47:53.418941 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407882 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 03:47:53.418941 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407884 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 03:47:53.418941 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407887 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 03:47:53.418941 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407890 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 03:47:53.418941 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407892 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 03:47:53.418941 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407898 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 03:47:53.418941 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407901 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 03:47:53.418941 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407904 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 03:47:53.418941 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407907 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 03:47:53.418941 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407909 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 03:47:53.418941 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407912 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 03:47:53.418941 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407914 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 03:47:53.418941 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407917 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 03:47:53.418941 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407920 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 03:47:53.418941 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407922 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 03:47:53.418941 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407925 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 03:47:53.419422 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407928 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 03:47:53.419422 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407931 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 03:47:53.419422 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.407933 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 03:47:53.419422 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.408637 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 03:47:53.419422 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.417405 2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 10 03:47:53.419422 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.417425 2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 10 03:47:53.419422 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.418176 2562 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 10 03:47:53.419422 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.418184 2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 10 03:47:53.419422 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418245 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 03:47:53.419422 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418251 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 03:47:53.419422 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418255 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 03:47:53.419422 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418258 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 03:47:53.419422 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418261 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 03:47:53.419422 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418263 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 03:47:53.419766 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418266 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 03:47:53.419766 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418269 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 03:47:53.419766 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418272 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 03:47:53.419766 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418274 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 03:47:53.419766 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418277 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 03:47:53.419766 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418280 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 03:47:53.419766 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418283 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 03:47:53.419766 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418286 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 03:47:53.419766 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418303 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 03:47:53.419766 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418307 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 03:47:53.419766 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418309 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 03:47:53.419766 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418312 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 03:47:53.419766 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418315 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 03:47:53.419766 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418318 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 03:47:53.419766 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418321 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 03:47:53.419766 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418323 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 03:47:53.419766 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418326 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 03:47:53.419766 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418329 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 03:47:53.419766 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418332 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 03:47:53.419766 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418334 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 03:47:53.420258 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418337 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 03:47:53.420258 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418341 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 03:47:53.420258 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418344 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 03:47:53.420258 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418347 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 03:47:53.420258 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418350 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 03:47:53.420258 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418353 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 03:47:53.420258 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418356 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 03:47:53.420258 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418359 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 03:47:53.420258 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418361 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 03:47:53.420258 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418364 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 03:47:53.420258 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418367 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 03:47:53.420258 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418369 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 03:47:53.420258 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418372 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 03:47:53.420258 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418375 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 03:47:53.420258 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418377 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 03:47:53.420258 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418380 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 03:47:53.420258 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418382 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 03:47:53.420258 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418385 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 03:47:53.420258 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418387 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 03:47:53.420258 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418390 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 03:47:53.420779 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418393 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 03:47:53.420779 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418396 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 03:47:53.420779 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418399 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 03:47:53.420779 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418401 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 03:47:53.420779 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418404 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 03:47:53.420779 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418407 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 03:47:53.420779 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418410 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 03:47:53.420779 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418415 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 03:47:53.420779 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418418 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 03:47:53.420779 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418421 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 03:47:53.420779 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418424 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 03:47:53.420779 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418426 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 03:47:53.420779 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418429 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 03:47:53.420779 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418431 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 03:47:53.420779 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418434 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 03:47:53.420779 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418437 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 03:47:53.420779 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418439 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 03:47:53.420779 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418442 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 03:47:53.420779 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418445 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 03:47:53.420779 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418447 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 03:47:53.421261 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418450 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 03:47:53.421261 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418452 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 03:47:53.421261 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418455 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 03:47:53.421261 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418458 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 03:47:53.421261 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418460 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 03:47:53.421261 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418464 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 03:47:53.421261 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418468 2562 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 03:47:53.421261 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418471 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 03:47:53.421261 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418474 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 03:47:53.421261 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418478 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 03:47:53.421261 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418481 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 03:47:53.421261 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418484 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 03:47:53.421261 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418487 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 03:47:53.421261 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418491 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 03:47:53.421261 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418494 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 03:47:53.421261 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418497 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 03:47:53.421261 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418499 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 03:47:53.421261 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418502 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 03:47:53.421261 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418505 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 03:47:53.421835 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418508 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 03:47:53.421835 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418510 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 03:47:53.421835 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418513 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 03:47:53.421835 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418516 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 03:47:53.421835 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418518 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 03:47:53.421835 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418521 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 03:47:53.421835 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418523 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 03:47:53.421835 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418526 2562 feature_gate.go:328] unrecognized feature gate: Example Sep 10 03:47:53.421835 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418529 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 03:47:53.421835 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418532 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 03:47:53.421835 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418534 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 03:47:53.421835 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418537 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 03:47:53.421835 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418540 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 03:47:53.421835 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418543 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 03:47:53.421835 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.418548 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 03:47:53.422216 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418662 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 03:47:53.422216 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418666 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 03:47:53.422216 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418670 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 03:47:53.422216 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418674 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 03:47:53.422216 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418677 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 03:47:53.422216 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418679 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 03:47:53.422216 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418682 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 03:47:53.422216 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418685 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 03:47:53.422216 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418688 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 03:47:53.422216 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418691 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 03:47:53.422216 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418694 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 03:47:53.422216 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418696 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 03:47:53.422216 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418701 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 03:47:53.422216 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418704 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 03:47:53.422216 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418707 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 03:47:53.422216 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418710 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 03:47:53.422216 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418713 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 03:47:53.422216 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418716 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 03:47:53.422216 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418719 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 03:47:53.422696 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418722 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 03:47:53.422696 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418724 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 03:47:53.422696 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418727 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 03:47:53.422696 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418730 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 03:47:53.422696 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418732 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 03:47:53.422696 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418735 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 03:47:53.422696 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418737 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 03:47:53.422696 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418740 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 03:47:53.422696 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418743 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 03:47:53.422696 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418745 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 03:47:53.422696 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418748 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 03:47:53.422696 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418751 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 03:47:53.422696 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418753 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 03:47:53.422696 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418756 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 03:47:53.422696 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418758 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 03:47:53.422696 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418761 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 03:47:53.422696 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418763 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 03:47:53.422696 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418766 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 03:47:53.422696 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418768 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 03:47:53.423159 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418771 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 03:47:53.423159 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418773 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 03:47:53.423159 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418776 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 03:47:53.423159 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418779 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 03:47:53.423159 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418781 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 03:47:53.423159 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418783 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 03:47:53.423159 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418786 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 03:47:53.423159 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418789 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 03:47:53.423159 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418791 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 03:47:53.423159 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418794 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 03:47:53.423159 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418797 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 03:47:53.423159 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418800 2562 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 03:47:53.423159 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418802 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 03:47:53.423159 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418804 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 03:47:53.423159 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418807 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 03:47:53.423159 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418810 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 03:47:53.423159 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418812 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 03:47:53.423159 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418815 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 03:47:53.423159 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418818 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 03:47:53.423159 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418820 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 03:47:53.423655 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418823 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 03:47:53.423655 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418826 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 03:47:53.423655 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418828 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 03:47:53.423655 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418831 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 03:47:53.423655 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418834 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 03:47:53.423655 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418837 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 03:47:53.423655 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418839 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 03:47:53.423655 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418842 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 03:47:53.423655 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418845 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 03:47:53.423655 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418847 2562 feature_gate.go:328] unrecognized feature gate: Example Sep 10 03:47:53.423655 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418850 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 03:47:53.423655 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418852 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 03:47:53.423655 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418854 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 03:47:53.423655 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418857 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 03:47:53.423655 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418859 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 03:47:53.423655 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418862 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 03:47:53.423655 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418864 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 03:47:53.423655 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418867 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 03:47:53.423655 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418869 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 03:47:53.423655 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418872 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 03:47:53.424122 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418876 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 03:47:53.424122 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418878 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 03:47:53.424122 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418881 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 03:47:53.424122 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418884 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 03:47:53.424122 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418887 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 03:47:53.424122 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418889 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 03:47:53.424122 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418892 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 03:47:53.424122 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418895 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 03:47:53.424122 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418897 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 03:47:53.424122 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418900 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 03:47:53.424122 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418903 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 03:47:53.424122 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418905 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 03:47:53.424122 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418908 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 03:47:53.424122 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418910 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 03:47:53.424122 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418913 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 03:47:53.424122 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418915 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 03:47:53.424122 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418918 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 03:47:53.424122 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418920 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 03:47:53.424122 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418923 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 03:47:53.424122 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418925 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 03:47:53.424618 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:53.418928 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 03:47:53.424618 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.418933 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 03:47:53.424618 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.419050 2562 server.go:962] "Client rotation is on, will bootstrap in background" Sep 10 03:47:53.424618 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.422742 2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 10 03:47:53.424618 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.424167 2562 server.go:1019] "Starting client certificate rotation" Sep 10 03:47:53.424618 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.424261 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 03:47:53.424618 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.424302 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 03:47:53.427699 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.427685 2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 10 03:47:53.428866 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.428847 2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 10 03:47:53.445696 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.445672 2562 log.go:25] "Validated CRI v1 runtime API" Sep 10 03:47:53.448764 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.448749 2562 log.go:25] "Validated CRI v1 image API" Sep 10 03:47:53.449650 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.449633 2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 10 03:47:53.452891 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.452867 2562 fs.go:135] Filesystem UUIDs: map[5070f3aa-b484-4e35-b991-c0a54da4ac55:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 955bcff3-c0e2-40e6-ae06-5a7889ca45ea:/dev/nvme0n1p4] Sep 10 03:47:53.452986 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.452888 2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 10 03:47:53.455218 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.455202 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 10 03:47:53.458057 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.457943 2562 manager.go:216] Machine: {Timestamp:2026-09-10 03:47:53.456717646 +0000 UTC m=+0.332780789 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3213137 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26126405117e744615ec7fb945d3fa SystemUUID:ec261264-0511-7e74-4615-ec7fb945d3fa BootID:5965d598-b89e-486c-a01d-3d9162ca7ba7 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:04:7f:06:16:a7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:04:7f:06:16:a7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:da:d4:ea:86:7a:39 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 10 03:47:53.458057 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.458056 2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 10 03:47:53.458160 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.458148 2562 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 10 03:47:53.459867 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.459843 2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 10 03:47:53.460011 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.459874 2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-74.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 10 03:47:53.460592 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.460582 2562 topology_manager.go:138] "Creating topology manager with none policy" Sep 10 03:47:53.460623 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.460594 2562 container_manager_linux.go:309] "Creating device plugin manager" Sep 10 03:47:53.460623 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.460602 2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 10 03:47:53.461317 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.461306 2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 10 03:47:53.462742 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.462732 2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 10 03:47:53.466751 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.466740 2562 state_mem.go:36] "Initialized new in-memory state store" Sep 10 03:47:53.466872 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.466861 2562 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 10 03:47:53.466909 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.466895 2562 kubelet.go:486] "Attempting to sync node with API server" Sep 10 03:47:53.466940 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.466910 2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 10 03:47:53.466940 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.466934 2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 10 03:47:53.467001 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.466948 2562 kubelet.go:398] "Adding apiserver pod source" Sep 10 03:47:53.467001 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.466959 2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 10 03:47:53.467465 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.467440 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 10 03:47:53.467465 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.467466 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 10 03:47:53.473390 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.473366 2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 10 03:47:53.476225 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.476209 2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 10 03:47:53.476336 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.476239 2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 10 03:47:53.477111 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.477099 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 10 03:47:53.477159 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.477116 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 10 03:47:53.477159 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.477122 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 10 03:47:53.477159 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.477127 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 10 03:47:53.477159 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.477136 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 10 03:47:53.477159 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.477141 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 10 03:47:53.477159 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.477155 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 10 03:47:53.477159 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.477160 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 10 03:47:53.477370 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.477166 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 10 03:47:53.477370 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.477173 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 10 03:47:53.477370 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.477181 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 10 03:47:53.477370 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.477186 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 10 03:47:53.477974 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.477961 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 10 03:47:53.477974 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.477970 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 10 03:47:53.478440 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:53.478408 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 10 03:47:53.478510 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:53.478442 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-74.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 10 03:47:53.481995 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.481825 2562 server.go:1268] "Started kubelet" Sep 10 03:47:53.482066 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.481909 2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 10 03:47:53.482116 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.482083 2562 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 10 03:47:53.482171 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.481916 2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 10 03:47:53.482868 ip-10-0-132-74 systemd[1]: Started Kubernetes Kubelet. Sep 10 03:47:53.483610 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.483592 2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 10 03:47:53.484317 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.484268 2562 server.go:310] "Adding debug handlers to kubelet server" Sep 10 03:47:53.488867 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:53.488838 2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 10 03:47:53.489546 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.489528 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 10 03:47:53.489630 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.489528 2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 10 03:47:53.489744 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:53.489660 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-74.ec2.internal\" not found" Sep 10 03:47:53.490276 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.490261 2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 10 03:47:53.490276 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.490262 2562 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 10 03:47:53.490436 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.490285 2562 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 10 03:47:53.490436 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.490403 2562 reconstruct.go:97] "Volume reconstruction finished" Sep 10 03:47:53.490436 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.490410 2562 reconciler.go:29] "Reconciler: start to sync state" Sep 10 03:47:53.490569 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.490536 2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-74.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 03:47:53.490651 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.490632 2562 factory.go:55] Registering systemd factory Sep 10 03:47:53.490651 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.490648 2562 factory.go:223] Registration of the systemd container factory successfully Sep 10 03:47:53.490872 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.490856 2562 factory.go:153] Registering CRI-O factory Sep 10 03:47:53.490942 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.490875 2562 factory.go:223] Registration of the crio container factory successfully Sep 10 03:47:53.490942 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.490922 2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 10 03:47:53.490942 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.490938 2562 factory.go:103] Registering Raw factory Sep 10 03:47:53.491090 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.490950 2562 manager.go:1196] Started watching for new ooms in manager Sep 10 03:47:53.491481 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.491451 2562 manager.go:349] Starting recovery of all containers Sep 10 03:47:53.491559 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:53.490497 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-74.ec2.internal.18d3d90092c47788 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-74.ec2.internal,UID:ip-10-0-132-74.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-74.ec2.internal,},FirstTimestamp:2026-09-10 03:47:53.481795464 +0000 UTC m=+0.357858609,LastTimestamp:2026-09-10 03:47:53.481795464 +0000 UTC m=+0.357858609,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-74.ec2.internal,}" Sep 10 03:47:53.494371 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:53.494344 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 10 03:47:53.494477 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:53.494446 2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-74.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 10 03:47:53.497451 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.497423 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 10 03:47:53.503059 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.503009 2562 manager.go:354] Recovery completed Sep 10 03:47:53.503777 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.503753 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vdq9h" Sep 10 03:47:53.508825 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.508813 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 03:47:53.510578 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.510563 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-74.ec2.internal" event="NodeHasSufficientMemory" Sep 10 03:47:53.510651 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.510591 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-74.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 03:47:53.510651 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.510604 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-74.ec2.internal" event="NodeHasSufficientPID" Sep 10 03:47:53.511067 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.511048 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vdq9h" Sep 10 03:47:53.511150 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.511134 2562 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 10 03:47:53.511150 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.511147 2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 10 03:47:53.511241 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.511153 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 03:47:53.511241 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.511164 2562 state_mem.go:36] "Initialized new in-memory state store" Sep 10 03:47:53.511802 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.511785 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-74.ec2.internal" event="NodeHasSufficientMemory" Sep 10 03:47:53.511873 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.511811 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-74.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 03:47:53.511873 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.511821 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-74.ec2.internal" event="NodeHasSufficientPID" Sep 10 03:47:53.512474 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:53.512398 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-74.ec2.internal.18d3d900947ba642 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-74.ec2.internal,UID:ip-10-0-132-74.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-74.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-74.ec2.internal,},FirstTimestamp:2026-09-10 03:47:53.51057773 +0000 UTC m=+0.386640873,LastTimestamp:2026-09-10 03:47:53.51057773 +0000 UTC m=+0.386640873,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-74.ec2.internal,}" Sep 10 03:47:53.513225 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.513215 2562 policy_none.go:49] "None policy: Start" Sep 10 03:47:53.513264 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.513230 2562 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 10 03:47:53.513264 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.513240 2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 10 03:47:53.590564 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:53.590527 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-74.ec2.internal\" not found" Sep 10 03:47:53.657108 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.597929 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 10 03:47:53.657108 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.597989 2562 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 10 03:47:53.657108 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.598018 2562 kubelet.go:2448] "Starting kubelet main sync loop" Sep 10 03:47:53.657108 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:53.598054 2562 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Sep 10 03:47:53.657108 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.600444 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 03:47:53.657108 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.656053 2562 policy_none.go:47] "Start" Sep 10 03:47:53.691454 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:53.691431 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-74.ec2.internal\" not found" Sep 10 03:47:53.698221 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:53.698206 2562 kubelet.go:2472] "Skipping pod synchronization" err="container runtime status check may not have completed yet" Sep 10 03:47:53.699433 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:53.699420 2562 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-132-74.ec2.internal\" not found" node="ip-10-0-132-74.ec2.internal" Sep 10 03:47:53.705011 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.704997 2562 manager.go:337] "Starting Device Plugin manager" Sep 10 03:47:53.705096 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:53.705057 2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 10 03:47:53.705096 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.705070 2562 server.go:87] "Starting device plugin registration server" Sep 10 03:47:53.705488 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.705473 2562 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 10 03:47:53.705560 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.705488 2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 10 03:47:53.705638 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.705603 2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 10 03:47:53.705709 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.705702 2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 10 03:47:53.705762 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.705712 2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 10 03:47:53.706273 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:53.706253 2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 10 03:47:53.706368 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:53.706328 2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-74.ec2.internal\" not found" Sep 10 03:47:53.805906 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.805874 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 03:47:53.808450 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.808432 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-74.ec2.internal" event="NodeHasSufficientMemory" Sep 10 03:47:53.808538 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.808461 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-74.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 03:47:53.808538 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.808471 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-74.ec2.internal" event="NodeHasSufficientPID" Sep 10 03:47:53.808538 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.808493 2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-74.ec2.internal" Sep 10 03:47:53.816450 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.816434 2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-74.ec2.internal" Sep 10 03:47:53.816495 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:53.816458 2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-74.ec2.internal\": node \"ip-10-0-132-74.ec2.internal\" not found" Sep 10 03:47:53.843822 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:53.843796 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-74.ec2.internal\" not found" Sep 10 03:47:53.899173 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.899132 2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-74.ec2.internal"] Sep 10 03:47:53.899366 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.899227 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 03:47:53.900194 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.900178 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-74.ec2.internal" event="NodeHasSufficientMemory" Sep 10 03:47:53.900269 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.900208 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-74.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 03:47:53.900269 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.900218 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-74.ec2.internal" event="NodeHasSufficientPID" Sep 10 03:47:53.902576 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.902564 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 03:47:53.902682 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.902667 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.ec2.internal" Sep 10 03:47:53.902720 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.902696 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 03:47:53.903373 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.903353 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-74.ec2.internal" event="NodeHasSufficientMemory" Sep 10 03:47:53.903468 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.903389 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-74.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 03:47:53.903468 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.903402 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-74.ec2.internal" event="NodeHasSufficientPID" Sep 10 03:47:53.904202 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.904184 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-74.ec2.internal" event="NodeHasSufficientMemory" Sep 10 03:47:53.904284 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.904212 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-74.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 03:47:53.904284 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.904224 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-74.ec2.internal" event="NodeHasSufficientPID" Sep 10 03:47:53.905527 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.905514 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-74.ec2.internal" Sep 10 03:47:53.905582 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.905538 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 03:47:53.906248 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.906233 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-74.ec2.internal" event="NodeHasSufficientMemory" Sep 10 03:47:53.906328 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.906260 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-74.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 03:47:53.906328 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.906274 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-74.ec2.internal" event="NodeHasSufficientPID" Sep 10 03:47:53.923916 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:53.923888 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-74.ec2.internal\" not found" node="ip-10-0-132-74.ec2.internal" Sep 10 03:47:53.927457 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:53.927440 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-74.ec2.internal\" not found" node="ip-10-0-132-74.ec2.internal" Sep 10 03:47:53.944541 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:53.944516 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-74.ec2.internal\" not found" Sep 10 03:47:53.993096 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.993067 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/95b230e05074eeabdbb71cf81104af8d-config\") pod \"kube-apiserver-proxy-ip-10-0-132-74.ec2.internal\" (UID: \"95b230e05074eeabdbb71cf81104af8d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-74.ec2.internal" Sep 10 03:47:53.993200 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.993099 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d8b7ce27f94ad4477741ec9d86dc3b51-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-74.ec2.internal\" (UID: \"d8b7ce27f94ad4477741ec9d86dc3b51\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.ec2.internal" Sep 10 03:47:53.993200 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:53.993117 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8b7ce27f94ad4477741ec9d86dc3b51-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-74.ec2.internal\" (UID: \"d8b7ce27f94ad4477741ec9d86dc3b51\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.ec2.internal" Sep 10 03:47:54.045220 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:54.045186 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-74.ec2.internal\" not found" Sep 10 03:47:54.093413 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:54.093384 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d8b7ce27f94ad4477741ec9d86dc3b51-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-74.ec2.internal\" (UID: \"d8b7ce27f94ad4477741ec9d86dc3b51\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.ec2.internal" Sep 10 03:47:54.093506 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:54.093416 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8b7ce27f94ad4477741ec9d86dc3b51-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-74.ec2.internal\" (UID: \"d8b7ce27f94ad4477741ec9d86dc3b51\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.ec2.internal" Sep 10 03:47:54.093506 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:54.093437 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/95b230e05074eeabdbb71cf81104af8d-config\") pod \"kube-apiserver-proxy-ip-10-0-132-74.ec2.internal\" (UID: \"95b230e05074eeabdbb71cf81104af8d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-74.ec2.internal" Sep 10 03:47:54.093506 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:54.093476 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8b7ce27f94ad4477741ec9d86dc3b51-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-74.ec2.internal\" (UID: \"d8b7ce27f94ad4477741ec9d86dc3b51\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.ec2.internal" Sep 10 03:47:54.093506 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:54.093485 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d8b7ce27f94ad4477741ec9d86dc3b51-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-74.ec2.internal\" (UID: \"d8b7ce27f94ad4477741ec9d86dc3b51\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.ec2.internal" Sep 10 03:47:54.093632 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:54.093524 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/95b230e05074eeabdbb71cf81104af8d-config\") pod \"kube-apiserver-proxy-ip-10-0-132-74.ec2.internal\" (UID: \"95b230e05074eeabdbb71cf81104af8d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-74.ec2.internal" Sep 10 03:47:54.148711 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:54.148670 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-74.ec2.internal\" not found" Sep 10 03:47:54.228330 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:54.228230 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.ec2.internal" Sep 10 03:47:54.230560 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:54.230544 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-74.ec2.internal" Sep 10 03:47:54.249739 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:54.249707 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-74.ec2.internal\" not found" Sep 10 03:47:54.329269 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:54.329236 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 03:47:54.350509 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:54.350469 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-74.ec2.internal\" not found" Sep 10 03:47:54.414849 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:54.414812 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 03:47:54.423663 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:54.423640 2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 10 03:47:54.423827 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:54.423806 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 10 03:47:54.423888 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:54.423816 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 10 03:47:54.423888 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:54.423816 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 10 03:47:54.450754 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:54.450706 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-74.ec2.internal\" not found" Sep 10 03:47:54.513869 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:54.513757 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 03:42:53 +0000 UTC" deadline="2028-04-03 14:22:08.71575918 +0000 UTC" Sep 10 03:47:54.513869 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:54.513811 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13714h34m14.20195332s" Sep 10 03:47:54.552444 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:54.552399 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-74.ec2.internal\" not found" Sep 10 03:47:54.655554 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:54.655512 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-74.ec2.internal\" not found" Sep 10 03:47:54.755172 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:54.755141 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 03:47:54.791508 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:54.791387 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-74.ec2.internal" Sep 10 03:47:54.801548 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:54.801524 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 10 03:47:54.802551 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:54.802535 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.ec2.internal" Sep 10 03:47:54.814357 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:54.814334 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 10 03:47:55.028521 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:55.028284 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95b230e05074eeabdbb71cf81104af8d.slice/crio-61b0965d683a69929806a1db108738aafe7c3a29afcd2a2baeeb10f045e38e21 WatchSource:0}: Error finding container 61b0965d683a69929806a1db108738aafe7c3a29afcd2a2baeeb10f045e38e21: Status 404 returned error can't find the container with id 61b0965d683a69929806a1db108738aafe7c3a29afcd2a2baeeb10f045e38e21 Sep 10 03:47:55.029456 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:55.029432 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8b7ce27f94ad4477741ec9d86dc3b51.slice/crio-0f609eb788ea0904bca8e4d5083ecfba14d55e238392e6130b2b71debd0650eb WatchSource:0}: Error finding container 0f609eb788ea0904bca8e4d5083ecfba14d55e238392e6130b2b71debd0650eb: Status 404 returned error can't find the container with id 0f609eb788ea0904bca8e4d5083ecfba14d55e238392e6130b2b71debd0650eb Sep 10 03:47:55.035782 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.035760 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 03:47:55.310586 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.310553 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 03:47:55.468832 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.468795 2562 apiserver.go:52] "Watching apiserver" Sep 10 03:47:55.476703 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.476665 2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 10 03:47:55.477051 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.477027 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-2p9b8","openshift-network-operator/iptables-alerter-8hwtj","openshift-ovn-kubernetes/ovnkube-node-fl7hj","openshift-dns/node-resolver-fwlqw","openshift-image-registry/node-ca-nmf2n","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.ec2.internal","openshift-multus/multus-additional-cni-plugins-ckqrf","openshift-multus/multus-vwrxj","openshift-network-diagnostics/network-check-target-hlv2m","kube-system/global-pull-secret-syncer-gwzk7","kube-system/konnectivity-agent-6ds5c","kube-system/kube-apiserver-proxy-ip-10-0-132-74.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkxrh","openshift-cluster-node-tuning-operator/tuned-8bb28"] Sep 10 03:47:55.481161 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.481141 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ckqrf" Sep 10 03:47:55.483562 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.483538 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-db8gj\"" Sep 10 03:47:55.483693 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.483545 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 10 03:47:55.483693 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.483681 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 10 03:47:55.483830 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.483807 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkxrh" Sep 10 03:47:55.484104 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.484076 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 10 03:47:55.484104 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.484089 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 10 03:47:55.485507 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.485485 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 10 03:47:55.485837 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.485816 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 10 03:47:55.486010 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.485860 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 10 03:47:55.486306 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.486284 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5qcs5\"" Sep 10 03:47:55.486424 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.486288 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 10 03:47:55.486653 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.486635 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.488949 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.488927 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fwlqw" Sep 10 03:47:55.489046 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.489007 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 10 03:47:55.489110 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.489045 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 10 03:47:55.489161 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.489123 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 10 03:47:55.489161 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.489153 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 10 03:47:55.489820 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.489802 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 10 03:47:55.489904 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.489820 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-pjs29\"" Sep 10 03:47:55.489904 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.489806 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 10 03:47:55.490944 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.490930 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 10 03:47:55.491384 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.491356 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 10 03:47:55.491620 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.491599 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-2n9gp\"" Sep 10 03:47:55.491722 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.491651 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nmf2n" Sep 10 03:47:55.491722 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.491663 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 10 03:47:55.493685 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.493665 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 10 03:47:55.493789 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.493774 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 10 03:47:55.493854 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.493775 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 10 03:47:55.494240 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.494223 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nvfb8\"" Sep 10 03:47:55.494349 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.494267 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.496427 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.496404 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 10 03:47:55.496717 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.496701 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 10 03:47:55.496819 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.496787 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-72khx\"" Sep 10 03:47:55.496900 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.496882 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hlv2m" Sep 10 03:47:55.496987 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:55.496959 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hlv2m" podUID="10447409-1646-48fb-a5c9-494670b7d232" Sep 10 03:47:55.499468 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.499448 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2p9b8" Sep 10 03:47:55.499568 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:55.499508 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2p9b8" podUID="158bac16-c713-47cd-91d7-9fc36d96f570" Sep 10 03:47:55.501044 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.501023 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 10 03:47:55.501467 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.501447 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aae7f13d-be0c-4bd9-af61-40fc05f144c0-system-cni-dir\") pod \"multus-additional-cni-plugins-ckqrf\" (UID: \"aae7f13d-be0c-4bd9-af61-40fc05f144c0\") " pod="openshift-multus/multus-additional-cni-plugins-ckqrf" Sep 10 03:47:55.501554 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.501474 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84ksq\" (UniqueName: \"kubernetes.io/projected/aae7f13d-be0c-4bd9-af61-40fc05f144c0-kube-api-access-84ksq\") pod \"multus-additional-cni-plugins-ckqrf\" (UID: \"aae7f13d-be0c-4bd9-af61-40fc05f144c0\") " pod="openshift-multus/multus-additional-cni-plugins-ckqrf" Sep 10 03:47:55.501554 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.501499 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/434f994a-63b7-4290-a72a-eb55456f4790-socket-dir\") pod \"aws-ebs-csi-driver-node-jkxrh\" (UID: \"434f994a-63b7-4290-a72a-eb55456f4790\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkxrh" Sep 10 03:47:55.501554 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.501514 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/434f994a-63b7-4290-a72a-eb55456f4790-etc-selinux\") pod \"aws-ebs-csi-driver-node-jkxrh\" (UID: \"434f994a-63b7-4290-a72a-eb55456f4790\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkxrh" Sep 10 03:47:55.501554 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.501529 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dkfc\" (UniqueName: \"kubernetes.io/projected/434f994a-63b7-4290-a72a-eb55456f4790-kube-api-access-4dkfc\") pod \"aws-ebs-csi-driver-node-jkxrh\" (UID: \"434f994a-63b7-4290-a72a-eb55456f4790\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkxrh" Sep 10 03:47:55.501554 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.501552 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-host-slash\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.501866 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.501575 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/460613b7-2cdf-4e57-8edf-06e7b72f790a-hosts-file\") pod \"node-resolver-fwlqw\" (UID: \"460613b7-2cdf-4e57-8edf-06e7b72f790a\") " pod="openshift-dns/node-resolver-fwlqw" Sep 10 03:47:55.501866 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.501601 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aae7f13d-be0c-4bd9-af61-40fc05f144c0-cnibin\") pod \"multus-additional-cni-plugins-ckqrf\" (UID: \"aae7f13d-be0c-4bd9-af61-40fc05f144c0\") " pod="openshift-multus/multus-additional-cni-plugins-ckqrf" Sep 10 03:47:55.501866 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.501666 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/aae7f13d-be0c-4bd9-af61-40fc05f144c0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ckqrf\" (UID: \"aae7f13d-be0c-4bd9-af61-40fc05f144c0\") " pod="openshift-multus/multus-additional-cni-plugins-ckqrf" Sep 10 03:47:55.501866 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.501717 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-node-log\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.501866 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.501746 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-host-cni-netd\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.501866 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.501795 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/460613b7-2cdf-4e57-8edf-06e7b72f790a-tmp-dir\") pod \"node-resolver-fwlqw\" (UID: \"460613b7-2cdf-4e57-8edf-06e7b72f790a\") " pod="openshift-dns/node-resolver-fwlqw" Sep 10 03:47:55.501866 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.501822 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7298q\" (UniqueName: \"kubernetes.io/projected/460613b7-2cdf-4e57-8edf-06e7b72f790a-kube-api-access-7298q\") pod \"node-resolver-fwlqw\" (UID: \"460613b7-2cdf-4e57-8edf-06e7b72f790a\") " pod="openshift-dns/node-resolver-fwlqw" Sep 10 03:47:55.501866 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.501846 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-host-run-ovn-kubernetes\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.501866 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.501865 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.502405 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.501903 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8459a996-8d64-4c23-9e92-1a5bcca253b5-ovnkube-config\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.502405 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.501934 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmj2c\" (UniqueName: \"kubernetes.io/projected/1ef729ce-06d8-4c1d-ac8c-7a89770df17f-kube-api-access-dmj2c\") pod \"node-ca-nmf2n\" (UID: \"1ef729ce-06d8-4c1d-ac8c-7a89770df17f\") " pod="openshift-image-registry/node-ca-nmf2n" Sep 10 03:47:55.502405 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.501958 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/aae7f13d-be0c-4bd9-af61-40fc05f144c0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ckqrf\" (UID: \"aae7f13d-be0c-4bd9-af61-40fc05f144c0\") " pod="openshift-multus/multus-additional-cni-plugins-ckqrf" Sep 10 03:47:55.502405 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.502021 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/434f994a-63b7-4290-a72a-eb55456f4790-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jkxrh\" (UID: \"434f994a-63b7-4290-a72a-eb55456f4790\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkxrh" Sep 10 03:47:55.502405 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.502045 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-etc-openvswitch\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.502405 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.502103 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-run-openvswitch\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.502405 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.502131 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/434f994a-63b7-4290-a72a-eb55456f4790-sys-fs\") pod \"aws-ebs-csi-driver-node-jkxrh\" (UID: \"434f994a-63b7-4290-a72a-eb55456f4790\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkxrh" Sep 10 03:47:55.502405 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.502155 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-host-kubelet\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.502405 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.502179 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8459a996-8d64-4c23-9e92-1a5bcca253b5-env-overrides\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.502405 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.502202 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8459a996-8d64-4c23-9e92-1a5bcca253b5-ovn-node-metrics-cert\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.502405 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.502248 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1ef729ce-06d8-4c1d-ac8c-7a89770df17f-serviceca\") pod \"node-ca-nmf2n\" (UID: \"1ef729ce-06d8-4c1d-ac8c-7a89770df17f\") " pod="openshift-image-registry/node-ca-nmf2n" Sep 10 03:47:55.502405 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.502265 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8hwtj" Sep 10 03:47:55.502405 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.502271 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-host-run-netns\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.502405 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.502296 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-run-ovn\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.502405 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.502352 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1ef729ce-06d8-4c1d-ac8c-7a89770df17f-host\") pod \"node-ca-nmf2n\" (UID: \"1ef729ce-06d8-4c1d-ac8c-7a89770df17f\") " pod="openshift-image-registry/node-ca-nmf2n" Sep 10 03:47:55.502405 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.502372 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aae7f13d-be0c-4bd9-af61-40fc05f144c0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ckqrf\" (UID: \"aae7f13d-be0c-4bd9-af61-40fc05f144c0\") " pod="openshift-multus/multus-additional-cni-plugins-ckqrf" Sep 10 03:47:55.502405 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.502387 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/434f994a-63b7-4290-a72a-eb55456f4790-device-dir\") pod \"aws-ebs-csi-driver-node-jkxrh\" (UID: \"434f994a-63b7-4290-a72a-eb55456f4790\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkxrh" Sep 10 03:47:55.503087 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.502409 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-run-systemd\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.503087 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.502456 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-log-socket\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.503087 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.502518 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8459a996-8d64-4c23-9e92-1a5bcca253b5-ovnkube-script-lib\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.503087 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.502546 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aae7f13d-be0c-4bd9-af61-40fc05f144c0-cni-binary-copy\") pod \"multus-additional-cni-plugins-ckqrf\" (UID: \"aae7f13d-be0c-4bd9-af61-40fc05f144c0\") " pod="openshift-multus/multus-additional-cni-plugins-ckqrf" Sep 10 03:47:55.503087 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.502592 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/434f994a-63b7-4290-a72a-eb55456f4790-registration-dir\") pod \"aws-ebs-csi-driver-node-jkxrh\" (UID: \"434f994a-63b7-4290-a72a-eb55456f4790\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkxrh" Sep 10 03:47:55.503087 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.502618 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-systemd-units\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.503087 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.502650 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-var-lib-openvswitch\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.503087 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.502675 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-host-cni-bin\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.503087 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.502701 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkq8t\" (UniqueName: \"kubernetes.io/projected/8459a996-8d64-4c23-9e92-1a5bcca253b5-kube-api-access-fkq8t\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.504576 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.504551 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 10 03:47:55.505094 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.504925 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 10 03:47:55.505094 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.504983 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gwzk7" Sep 10 03:47:55.505094 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.505068 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-dwkww\"" Sep 10 03:47:55.505094 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:55.505078 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gwzk7" podUID="348bacd3-9294-48e6-bbf4-60727b9a7bfb" Sep 10 03:47:55.505598 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.505578 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 10 03:47:55.509124 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.509102 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6ds5c" Sep 10 03:47:55.511380 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.511357 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 10 03:47:55.511498 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.511461 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 10 03:47:55.511498 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.511461 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jwtgd\"" Sep 10 03:47:55.511606 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.511494 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.513454 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.513433 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 10 03:47:55.513551 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.513456 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-c28mh\"" Sep 10 03:47:55.526559 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.526536 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-bhdmn" Sep 10 03:47:55.535447 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.535428 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-bhdmn" Sep 10 03:47:55.591949 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.591868 2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 10 03:47:55.602876 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.602841 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7298q\" (UniqueName: \"kubernetes.io/projected/460613b7-2cdf-4e57-8edf-06e7b72f790a-kube-api-access-7298q\") pod \"node-resolver-fwlqw\" (UID: \"460613b7-2cdf-4e57-8edf-06e7b72f790a\") " pod="openshift-dns/node-resolver-fwlqw" Sep 10 03:47:55.603026 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.602883 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhrqr\" (UniqueName: \"kubernetes.io/projected/10447409-1646-48fb-a5c9-494670b7d232-kube-api-access-rhrqr\") pod \"network-check-target-hlv2m\" (UID: \"10447409-1646-48fb-a5c9-494670b7d232\") " pod="openshift-network-diagnostics/network-check-target-hlv2m" Sep 10 03:47:55.603026 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.602917 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-multus-conf-dir\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.603026 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.602941 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ddad2c33-abd1-44e0-a8f0-1180786284b8-etc-sysconfig\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.603026 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.602966 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ddad2c33-abd1-44e0-a8f0-1180786284b8-etc-kubernetes\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.603026 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.602997 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aae7f13d-be0c-4bd9-af61-40fc05f144c0-cnibin\") pod \"multus-additional-cni-plugins-ckqrf\" (UID: \"aae7f13d-be0c-4bd9-af61-40fc05f144c0\") " pod="openshift-multus/multus-additional-cni-plugins-ckqrf" Sep 10 03:47:55.603026 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603020 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-node-log\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.603342 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603047 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-hostroot\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.603342 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603093 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-host-run-ovn-kubernetes\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.603342 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603094 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aae7f13d-be0c-4bd9-af61-40fc05f144c0-cnibin\") pod \"multus-additional-cni-plugins-ckqrf\" (UID: \"aae7f13d-be0c-4bd9-af61-40fc05f144c0\") " pod="openshift-multus/multus-additional-cni-plugins-ckqrf" Sep 10 03:47:55.603342 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603157 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8459a996-8d64-4c23-9e92-1a5bcca253b5-ovnkube-config\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.603342 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603158 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-host-run-ovn-kubernetes\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.603342 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603196 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-node-log\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.603342 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603232 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddad2c33-abd1-44e0-a8f0-1180786284b8-sys\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.603342 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603273 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ddad2c33-abd1-44e0-a8f0-1180786284b8-lib-modules\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.603342 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603308 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158bac16-c713-47cd-91d7-9fc36d96f570-metrics-certs\") pod \"network-metrics-daemon-2p9b8\" (UID: \"158bac16-c713-47cd-91d7-9fc36d96f570\") " pod="openshift-multus/network-metrics-daemon-2p9b8" Sep 10 03:47:55.603843 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603350 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/46180a14-d2d5-44da-8ed7-80a47696fcf7-agent-certs\") pod \"konnectivity-agent-6ds5c\" (UID: \"46180a14-d2d5-44da-8ed7-80a47696fcf7\") " pod="kube-system/konnectivity-agent-6ds5c" Sep 10 03:47:55.603843 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603384 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-run-openvswitch\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.603843 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603403 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-host-kubelet\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.603843 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603434 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-run-openvswitch\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.603843 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603447 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8459a996-8d64-4c23-9e92-1a5bcca253b5-env-overrides\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.603843 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603473 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ddad2c33-abd1-44e0-a8f0-1180786284b8-etc-sysctl-d\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.603843 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603450 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-host-kubelet\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.603843 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603489 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b0936f72-795e-4e46-8316-ecc03bb2dec6-cni-binary-copy\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.603843 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603511 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6c86e38b-1231-4756-9be9-89db8bf3af80-iptables-alerter-script\") pod \"iptables-alerter-8hwtj\" (UID: \"6c86e38b-1231-4756-9be9-89db8bf3af80\") " pod="openshift-network-operator/iptables-alerter-8hwtj" Sep 10 03:47:55.603843 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603536 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-run-ovn\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.603843 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603557 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnln8\" (UniqueName: \"kubernetes.io/projected/ddad2c33-abd1-44e0-a8f0-1180786284b8-kube-api-access-wnln8\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.603843 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603573 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-cnibin\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.603843 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603591 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfn49\" (UniqueName: \"kubernetes.io/projected/b0936f72-795e-4e46-8316-ecc03bb2dec6-kube-api-access-bfn49\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.603843 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603599 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-run-ovn\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.603843 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603619 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aae7f13d-be0c-4bd9-af61-40fc05f144c0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ckqrf\" (UID: \"aae7f13d-be0c-4bd9-af61-40fc05f144c0\") " pod="openshift-multus/multus-additional-cni-plugins-ckqrf" Sep 10 03:47:55.603843 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603652 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/434f994a-63b7-4290-a72a-eb55456f4790-device-dir\") pod \"aws-ebs-csi-driver-node-jkxrh\" (UID: \"434f994a-63b7-4290-a72a-eb55456f4790\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkxrh" Sep 10 03:47:55.603843 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603704 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/434f994a-63b7-4290-a72a-eb55456f4790-device-dir\") pod \"aws-ebs-csi-driver-node-jkxrh\" (UID: \"434f994a-63b7-4290-a72a-eb55456f4790\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkxrh" Sep 10 03:47:55.604734 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603715 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ddad2c33-abd1-44e0-a8f0-1180786284b8-var-lib-kubelet\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.604734 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603746 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ddad2c33-abd1-44e0-a8f0-1180786284b8-tmp\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.604734 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603781 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-system-cni-dir\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.604734 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603817 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-multus-socket-dir-parent\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.604734 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603845 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aae7f13d-be0c-4bd9-af61-40fc05f144c0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ckqrf\" (UID: \"aae7f13d-be0c-4bd9-af61-40fc05f144c0\") " pod="openshift-multus/multus-additional-cni-plugins-ckqrf" Sep 10 03:47:55.604734 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603850 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/434f994a-63b7-4290-a72a-eb55456f4790-registration-dir\") pod \"aws-ebs-csi-driver-node-jkxrh\" (UID: \"434f994a-63b7-4290-a72a-eb55456f4790\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkxrh" Sep 10 03:47:55.604734 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603893 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/434f994a-63b7-4290-a72a-eb55456f4790-registration-dir\") pod \"aws-ebs-csi-driver-node-jkxrh\" (UID: \"434f994a-63b7-4290-a72a-eb55456f4790\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkxrh" Sep 10 03:47:55.604734 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603885 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.ec2.internal" event={"ID":"d8b7ce27f94ad4477741ec9d86dc3b51","Type":"ContainerStarted","Data":"0f609eb788ea0904bca8e4d5083ecfba14d55e238392e6130b2b71debd0650eb"} Sep 10 03:47:55.604734 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603938 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-var-lib-openvswitch\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.604734 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603960 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-var-lib-openvswitch\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.604734 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603967 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-host-cni-bin\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.604734 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.603998 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-host-cni-bin\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.604734 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.604001 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkq8t\" (UniqueName: \"kubernetes.io/projected/8459a996-8d64-4c23-9e92-1a5bcca253b5-kube-api-access-fkq8t\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.604734 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.604027 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ddad2c33-abd1-44e0-a8f0-1180786284b8-host\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.604734 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.604053 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aae7f13d-be0c-4bd9-af61-40fc05f144c0-system-cni-dir\") pod \"multus-additional-cni-plugins-ckqrf\" (UID: \"aae7f13d-be0c-4bd9-af61-40fc05f144c0\") " pod="openshift-multus/multus-additional-cni-plugins-ckqrf" Sep 10 03:47:55.604734 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.604079 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84ksq\" (UniqueName: \"kubernetes.io/projected/aae7f13d-be0c-4bd9-af61-40fc05f144c0-kube-api-access-84ksq\") pod \"multus-additional-cni-plugins-ckqrf\" (UID: \"aae7f13d-be0c-4bd9-af61-40fc05f144c0\") " pod="openshift-multus/multus-additional-cni-plugins-ckqrf" Sep 10 03:47:55.604734 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.604118 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aae7f13d-be0c-4bd9-af61-40fc05f144c0-system-cni-dir\") pod \"multus-additional-cni-plugins-ckqrf\" (UID: \"aae7f13d-be0c-4bd9-af61-40fc05f144c0\") " pod="openshift-multus/multus-additional-cni-plugins-ckqrf" Sep 10 03:47:55.605565 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.604120 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dkfc\" (UniqueName: \"kubernetes.io/projected/434f994a-63b7-4290-a72a-eb55456f4790-kube-api-access-4dkfc\") pod \"aws-ebs-csi-driver-node-jkxrh\" (UID: \"434f994a-63b7-4290-a72a-eb55456f4790\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkxrh" Sep 10 03:47:55.605565 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.604296 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-host-slash\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.605565 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.604313 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ddad2c33-abd1-44e0-a8f0-1180786284b8-etc-modprobe-d\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.605565 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.604350 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8459a996-8d64-4c23-9e92-1a5bcca253b5-env-overrides\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.605565 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.604387 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-host-slash\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.605565 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.604406 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8459a996-8d64-4c23-9e92-1a5bcca253b5-ovnkube-config\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.605565 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.604358 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-multus-cni-dir\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.605565 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.604472 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-host-run-k8s-cni-cncf-io\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.605565 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.604500 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-host-var-lib-cni-bin\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.605565 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.604525 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ddad2c33-abd1-44e0-a8f0-1180786284b8-etc-systemd\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.605565 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.604555 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/aae7f13d-be0c-4bd9-af61-40fc05f144c0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ckqrf\" (UID: \"aae7f13d-be0c-4bd9-af61-40fc05f144c0\") " pod="openshift-multus/multus-additional-cni-plugins-ckqrf" Sep 10 03:47:55.605565 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.604582 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-host-cni-netd\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.605565 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.604608 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv48g\" (UniqueName: \"kubernetes.io/projected/6c86e38b-1231-4756-9be9-89db8bf3af80-kube-api-access-xv48g\") pod \"iptables-alerter-8hwtj\" (UID: \"6c86e38b-1231-4756-9be9-89db8bf3af80\") " pod="openshift-network-operator/iptables-alerter-8hwtj" Sep 10 03:47:55.605565 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.604633 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/348bacd3-9294-48e6-bbf4-60727b9a7bfb-dbus\") pod \"global-pull-secret-syncer-gwzk7\" (UID: \"348bacd3-9294-48e6-bbf4-60727b9a7bfb\") " pod="kube-system/global-pull-secret-syncer-gwzk7" Sep 10 03:47:55.605565 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.604653 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-host-cni-netd\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.605565 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.604660 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.605565 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.604688 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmj2c\" (UniqueName: \"kubernetes.io/projected/1ef729ce-06d8-4c1d-ac8c-7a89770df17f-kube-api-access-dmj2c\") pod \"node-ca-nmf2n\" (UID: \"1ef729ce-06d8-4c1d-ac8c-7a89770df17f\") " pod="openshift-image-registry/node-ca-nmf2n" Sep 10 03:47:55.606257 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.604715 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-host-var-lib-cni-multus\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.606257 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.604738 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-host-var-lib-kubelet\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.606257 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.604717 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.606257 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.604765 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/aae7f13d-be0c-4bd9-af61-40fc05f144c0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ckqrf\" (UID: \"aae7f13d-be0c-4bd9-af61-40fc05f144c0\") " pod="openshift-multus/multus-additional-cni-plugins-ckqrf" Sep 10 03:47:55.606257 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.604792 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/434f994a-63b7-4290-a72a-eb55456f4790-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jkxrh\" (UID: \"434f994a-63b7-4290-a72a-eb55456f4790\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkxrh" Sep 10 03:47:55.606257 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.604870 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-etc-openvswitch\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.606257 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.604875 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/434f994a-63b7-4290-a72a-eb55456f4790-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jkxrh\" (UID: \"434f994a-63b7-4290-a72a-eb55456f4790\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkxrh" Sep 10 03:47:55.606257 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.604958 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-etc-openvswitch\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.606257 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.604970 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ddad2c33-abd1-44e0-a8f0-1180786284b8-etc-sysctl-conf\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.606257 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605001 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fx4rm\" (UniqueName: \"kubernetes.io/projected/158bac16-c713-47cd-91d7-9fc36d96f570-kube-api-access-fx4rm\") pod \"network-metrics-daemon-2p9b8\" (UID: \"158bac16-c713-47cd-91d7-9fc36d96f570\") " pod="openshift-multus/network-metrics-daemon-2p9b8" Sep 10 03:47:55.606257 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605039 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/348bacd3-9294-48e6-bbf4-60727b9a7bfb-kubelet-config\") pod \"global-pull-secret-syncer-gwzk7\" (UID: \"348bacd3-9294-48e6-bbf4-60727b9a7bfb\") " pod="kube-system/global-pull-secret-syncer-gwzk7" Sep 10 03:47:55.606257 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605135 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/aae7f13d-be0c-4bd9-af61-40fc05f144c0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ckqrf\" (UID: \"aae7f13d-be0c-4bd9-af61-40fc05f144c0\") " pod="openshift-multus/multus-additional-cni-plugins-ckqrf" Sep 10 03:47:55.606257 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605141 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/434f994a-63b7-4290-a72a-eb55456f4790-sys-fs\") pod \"aws-ebs-csi-driver-node-jkxrh\" (UID: \"434f994a-63b7-4290-a72a-eb55456f4790\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkxrh" Sep 10 03:47:55.606257 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605165 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8459a996-8d64-4c23-9e92-1a5bcca253b5-ovn-node-metrics-cert\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.606257 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605196 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1ef729ce-06d8-4c1d-ac8c-7a89770df17f-serviceca\") pod \"node-ca-nmf2n\" (UID: \"1ef729ce-06d8-4c1d-ac8c-7a89770df17f\") " pod="openshift-image-registry/node-ca-nmf2n" Sep 10 03:47:55.606257 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605233 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-host-run-netns\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.606813 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605251 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/434f994a-63b7-4290-a72a-eb55456f4790-sys-fs\") pod \"aws-ebs-csi-driver-node-jkxrh\" (UID: \"434f994a-63b7-4290-a72a-eb55456f4790\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkxrh" Sep 10 03:47:55.606813 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605261 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-etc-kubernetes\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.606813 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605366 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/46180a14-d2d5-44da-8ed7-80a47696fcf7-konnectivity-ca\") pod \"konnectivity-agent-6ds5c\" (UID: \"46180a14-d2d5-44da-8ed7-80a47696fcf7\") " pod="kube-system/konnectivity-agent-6ds5c" Sep 10 03:47:55.606813 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605419 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-host-run-netns\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.606813 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605445 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1ef729ce-06d8-4c1d-ac8c-7a89770df17f-host\") pod \"node-ca-nmf2n\" (UID: \"1ef729ce-06d8-4c1d-ac8c-7a89770df17f\") " pod="openshift-image-registry/node-ca-nmf2n" Sep 10 03:47:55.606813 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605470 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ddad2c33-abd1-44e0-a8f0-1180786284b8-run\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.606813 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605469 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/aae7f13d-be0c-4bd9-af61-40fc05f144c0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ckqrf\" (UID: \"aae7f13d-be0c-4bd9-af61-40fc05f144c0\") " pod="openshift-multus/multus-additional-cni-plugins-ckqrf" Sep 10 03:47:55.606813 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605499 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/348bacd3-9294-48e6-bbf4-60727b9a7bfb-original-pull-secret\") pod \"global-pull-secret-syncer-gwzk7\" (UID: \"348bacd3-9294-48e6-bbf4-60727b9a7bfb\") " pod="kube-system/global-pull-secret-syncer-gwzk7" Sep 10 03:47:55.606813 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605512 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1ef729ce-06d8-4c1d-ac8c-7a89770df17f-host\") pod \"node-ca-nmf2n\" (UID: \"1ef729ce-06d8-4c1d-ac8c-7a89770df17f\") " pod="openshift-image-registry/node-ca-nmf2n" Sep 10 03:47:55.606813 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605513 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-74.ec2.internal" event={"ID":"95b230e05074eeabdbb71cf81104af8d","Type":"ContainerStarted","Data":"61b0965d683a69929806a1db108738aafe7c3a29afcd2a2baeeb10f045e38e21"} Sep 10 03:47:55.606813 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605525 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-run-systemd\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.606813 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605546 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-host-run-netns\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.606813 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605552 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-log-socket\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.606813 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605578 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8459a996-8d64-4c23-9e92-1a5bcca253b5-ovnkube-script-lib\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.606813 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605592 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-run-systemd\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.606813 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605601 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ddad2c33-abd1-44e0-a8f0-1180786284b8-etc-tuned\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.606813 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605641 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-log-socket\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.607344 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605631 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b0936f72-795e-4e46-8316-ecc03bb2dec6-multus-daemon-config\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.607344 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605675 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-host-run-multus-certs\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.607344 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605704 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1ef729ce-06d8-4c1d-ac8c-7a89770df17f-serviceca\") pod \"node-ca-nmf2n\" (UID: \"1ef729ce-06d8-4c1d-ac8c-7a89770df17f\") " pod="openshift-image-registry/node-ca-nmf2n" Sep 10 03:47:55.607344 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605705 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aae7f13d-be0c-4bd9-af61-40fc05f144c0-cni-binary-copy\") pod \"multus-additional-cni-plugins-ckqrf\" (UID: \"aae7f13d-be0c-4bd9-af61-40fc05f144c0\") " pod="openshift-multus/multus-additional-cni-plugins-ckqrf" Sep 10 03:47:55.607344 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605748 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-systemd-units\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.607344 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605768 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6c86e38b-1231-4756-9be9-89db8bf3af80-host-slash\") pod \"iptables-alerter-8hwtj\" (UID: \"6c86e38b-1231-4756-9be9-89db8bf3af80\") " pod="openshift-network-operator/iptables-alerter-8hwtj" Sep 10 03:47:55.607344 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605785 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/434f994a-63b7-4290-a72a-eb55456f4790-socket-dir\") pod \"aws-ebs-csi-driver-node-jkxrh\" (UID: \"434f994a-63b7-4290-a72a-eb55456f4790\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkxrh" Sep 10 03:47:55.607344 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605807 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/434f994a-63b7-4290-a72a-eb55456f4790-etc-selinux\") pod \"aws-ebs-csi-driver-node-jkxrh\" (UID: \"434f994a-63b7-4290-a72a-eb55456f4790\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkxrh" Sep 10 03:47:55.607344 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605810 2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 10 03:47:55.607344 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605835 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/460613b7-2cdf-4e57-8edf-06e7b72f790a-hosts-file\") pod \"node-resolver-fwlqw\" (UID: \"460613b7-2cdf-4e57-8edf-06e7b72f790a\") " pod="openshift-dns/node-resolver-fwlqw" Sep 10 03:47:55.607344 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605842 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8459a996-8d64-4c23-9e92-1a5bcca253b5-systemd-units\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.607344 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605860 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/460613b7-2cdf-4e57-8edf-06e7b72f790a-tmp-dir\") pod \"node-resolver-fwlqw\" (UID: \"460613b7-2cdf-4e57-8edf-06e7b72f790a\") " pod="openshift-dns/node-resolver-fwlqw" Sep 10 03:47:55.607344 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605922 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/434f994a-63b7-4290-a72a-eb55456f4790-socket-dir\") pod \"aws-ebs-csi-driver-node-jkxrh\" (UID: \"434f994a-63b7-4290-a72a-eb55456f4790\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkxrh" Sep 10 03:47:55.607344 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605946 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/460613b7-2cdf-4e57-8edf-06e7b72f790a-hosts-file\") pod \"node-resolver-fwlqw\" (UID: \"460613b7-2cdf-4e57-8edf-06e7b72f790a\") " pod="openshift-dns/node-resolver-fwlqw" Sep 10 03:47:55.607344 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.605945 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/434f994a-63b7-4290-a72a-eb55456f4790-etc-selinux\") pod \"aws-ebs-csi-driver-node-jkxrh\" (UID: \"434f994a-63b7-4290-a72a-eb55456f4790\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkxrh" Sep 10 03:47:55.607344 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.606138 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/460613b7-2cdf-4e57-8edf-06e7b72f790a-tmp-dir\") pod \"node-resolver-fwlqw\" (UID: \"460613b7-2cdf-4e57-8edf-06e7b72f790a\") " pod="openshift-dns/node-resolver-fwlqw" Sep 10 03:47:55.607344 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.606173 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8459a996-8d64-4c23-9e92-1a5bcca253b5-ovnkube-script-lib\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.607344 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.606179 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aae7f13d-be0c-4bd9-af61-40fc05f144c0-cni-binary-copy\") pod \"multus-additional-cni-plugins-ckqrf\" (UID: \"aae7f13d-be0c-4bd9-af61-40fc05f144c0\") " pod="openshift-multus/multus-additional-cni-plugins-ckqrf" Sep 10 03:47:55.609606 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.609585 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8459a996-8d64-4c23-9e92-1a5bcca253b5-ovn-node-metrics-cert\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.615035 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.615011 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7298q\" (UniqueName: \"kubernetes.io/projected/460613b7-2cdf-4e57-8edf-06e7b72f790a-kube-api-access-7298q\") pod \"node-resolver-fwlqw\" (UID: \"460613b7-2cdf-4e57-8edf-06e7b72f790a\") " pod="openshift-dns/node-resolver-fwlqw" Sep 10 03:47:55.619609 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.619356 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmj2c\" (UniqueName: \"kubernetes.io/projected/1ef729ce-06d8-4c1d-ac8c-7a89770df17f-kube-api-access-dmj2c\") pod \"node-ca-nmf2n\" (UID: \"1ef729ce-06d8-4c1d-ac8c-7a89770df17f\") " pod="openshift-image-registry/node-ca-nmf2n" Sep 10 03:47:55.620929 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.620904 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dkfc\" (UniqueName: \"kubernetes.io/projected/434f994a-63b7-4290-a72a-eb55456f4790-kube-api-access-4dkfc\") pod \"aws-ebs-csi-driver-node-jkxrh\" (UID: \"434f994a-63b7-4290-a72a-eb55456f4790\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkxrh" Sep 10 03:47:55.621609 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.621592 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkq8t\" (UniqueName: \"kubernetes.io/projected/8459a996-8d64-4c23-9e92-1a5bcca253b5-kube-api-access-fkq8t\") pod \"ovnkube-node-fl7hj\" (UID: \"8459a996-8d64-4c23-9e92-1a5bcca253b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.621699 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.621681 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84ksq\" (UniqueName: \"kubernetes.io/projected/aae7f13d-be0c-4bd9-af61-40fc05f144c0-kube-api-access-84ksq\") pod \"multus-additional-cni-plugins-ckqrf\" (UID: \"aae7f13d-be0c-4bd9-af61-40fc05f144c0\") " pod="openshift-multus/multus-additional-cni-plugins-ckqrf" Sep 10 03:47:55.707150 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.707108 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-hostroot\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.707150 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.707160 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddad2c33-abd1-44e0-a8f0-1180786284b8-sys\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.707419 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.707186 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ddad2c33-abd1-44e0-a8f0-1180786284b8-lib-modules\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.707419 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.707203 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-hostroot\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.707419 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.707212 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158bac16-c713-47cd-91d7-9fc36d96f570-metrics-certs\") pod \"network-metrics-daemon-2p9b8\" (UID: \"158bac16-c713-47cd-91d7-9fc36d96f570\") " pod="openshift-multus/network-metrics-daemon-2p9b8" Sep 10 03:47:55.707419 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.707265 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/46180a14-d2d5-44da-8ed7-80a47696fcf7-agent-certs\") pod \"konnectivity-agent-6ds5c\" (UID: \"46180a14-d2d5-44da-8ed7-80a47696fcf7\") " pod="kube-system/konnectivity-agent-6ds5c" Sep 10 03:47:55.707419 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.707291 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ddad2c33-abd1-44e0-a8f0-1180786284b8-etc-sysctl-d\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.707419 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.707316 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b0936f72-795e-4e46-8316-ecc03bb2dec6-cni-binary-copy\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.707419 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.707355 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6c86e38b-1231-4756-9be9-89db8bf3af80-iptables-alerter-script\") pod \"iptables-alerter-8hwtj\" (UID: \"6c86e38b-1231-4756-9be9-89db8bf3af80\") " pod="openshift-network-operator/iptables-alerter-8hwtj" Sep 10 03:47:55.707419 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.707376 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ddad2c33-abd1-44e0-a8f0-1180786284b8-lib-modules\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.707419 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.707384 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnln8\" (UniqueName: \"kubernetes.io/projected/ddad2c33-abd1-44e0-a8f0-1180786284b8-kube-api-access-wnln8\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.707419 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.707408 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-cnibin\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.707780 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.707430 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddad2c33-abd1-44e0-a8f0-1180786284b8-sys\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.707780 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.707434 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfn49\" (UniqueName: \"kubernetes.io/projected/b0936f72-795e-4e46-8316-ecc03bb2dec6-kube-api-access-bfn49\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.707780 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.707461 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ddad2c33-abd1-44e0-a8f0-1180786284b8-var-lib-kubelet\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.707780 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.707482 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ddad2c33-abd1-44e0-a8f0-1180786284b8-tmp\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.707780 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.707503 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-system-cni-dir\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.707780 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.707527 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-multus-socket-dir-parent\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.707780 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.707552 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ddad2c33-abd1-44e0-a8f0-1180786284b8-host\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.707780 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.707578 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ddad2c33-abd1-44e0-a8f0-1180786284b8-etc-modprobe-d\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.707780 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.707602 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-multus-cni-dir\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.707780 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.707634 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-host-run-k8s-cni-cncf-io\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.707780 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.707658 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-host-var-lib-cni-bin\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.707780 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.707686 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ddad2c33-abd1-44e0-a8f0-1180786284b8-etc-systemd\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.707780 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.707730 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xv48g\" (UniqueName: \"kubernetes.io/projected/6c86e38b-1231-4756-9be9-89db8bf3af80-kube-api-access-xv48g\") pod \"iptables-alerter-8hwtj\" (UID: \"6c86e38b-1231-4756-9be9-89db8bf3af80\") " pod="openshift-network-operator/iptables-alerter-8hwtj" Sep 10 03:47:55.707780 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.707755 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/348bacd3-9294-48e6-bbf4-60727b9a7bfb-dbus\") pod \"global-pull-secret-syncer-gwzk7\" (UID: \"348bacd3-9294-48e6-bbf4-60727b9a7bfb\") " pod="kube-system/global-pull-secret-syncer-gwzk7" Sep 10 03:47:55.707780 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.707781 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-host-var-lib-cni-multus\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.708218 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.707806 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-host-var-lib-kubelet\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.708218 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.707837 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ddad2c33-abd1-44e0-a8f0-1180786284b8-etc-sysctl-conf\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.708218 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.707866 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fx4rm\" (UniqueName: \"kubernetes.io/projected/158bac16-c713-47cd-91d7-9fc36d96f570-kube-api-access-fx4rm\") pod \"network-metrics-daemon-2p9b8\" (UID: \"158bac16-c713-47cd-91d7-9fc36d96f570\") " pod="openshift-multus/network-metrics-daemon-2p9b8" Sep 10 03:47:55.708218 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.707893 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/348bacd3-9294-48e6-bbf4-60727b9a7bfb-kubelet-config\") pod \"global-pull-secret-syncer-gwzk7\" (UID: \"348bacd3-9294-48e6-bbf4-60727b9a7bfb\") " pod="kube-system/global-pull-secret-syncer-gwzk7" Sep 10 03:47:55.708218 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.707925 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-host-run-netns\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.708218 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.707952 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-etc-kubernetes\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.708218 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.707978 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/46180a14-d2d5-44da-8ed7-80a47696fcf7-konnectivity-ca\") pod \"konnectivity-agent-6ds5c\" (UID: \"46180a14-d2d5-44da-8ed7-80a47696fcf7\") " pod="kube-system/konnectivity-agent-6ds5c" Sep 10 03:47:55.708218 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.707998 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b0936f72-795e-4e46-8316-ecc03bb2dec6-cni-binary-copy\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.708218 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.708005 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ddad2c33-abd1-44e0-a8f0-1180786284b8-run\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.708218 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.708029 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/348bacd3-9294-48e6-bbf4-60727b9a7bfb-original-pull-secret\") pod \"global-pull-secret-syncer-gwzk7\" (UID: \"348bacd3-9294-48e6-bbf4-60727b9a7bfb\") " pod="kube-system/global-pull-secret-syncer-gwzk7" Sep 10 03:47:55.708218 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.708058 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ddad2c33-abd1-44e0-a8f0-1180786284b8-etc-tuned\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.708218 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.708093 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b0936f72-795e-4e46-8316-ecc03bb2dec6-multus-daemon-config\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.708218 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:55.708110 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 03:47:55.708218 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.708119 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-host-run-multus-certs\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.708218 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.708148 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6c86e38b-1231-4756-9be9-89db8bf3af80-host-slash\") pod \"iptables-alerter-8hwtj\" (UID: \"6c86e38b-1231-4756-9be9-89db8bf3af80\") " pod="openshift-network-operator/iptables-alerter-8hwtj" Sep 10 03:47:55.708218 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:55.708189 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/158bac16-c713-47cd-91d7-9fc36d96f570-metrics-certs podName:158bac16-c713-47cd-91d7-9fc36d96f570 nodeName:}" failed. No retries permitted until 2026-09-10 03:47:56.208166926 +0000 UTC m=+3.084230084 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/158bac16-c713-47cd-91d7-9fc36d96f570-metrics-certs") pod "network-metrics-daemon-2p9b8" (UID: "158bac16-c713-47cd-91d7-9fc36d96f570") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 03:47:55.708218 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.708209 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhrqr\" (UniqueName: \"kubernetes.io/projected/10447409-1646-48fb-a5c9-494670b7d232-kube-api-access-rhrqr\") pod \"network-check-target-hlv2m\" (UID: \"10447409-1646-48fb-a5c9-494670b7d232\") " pod="openshift-network-diagnostics/network-check-target-hlv2m" Sep 10 03:47:55.708738 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.708214 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6c86e38b-1231-4756-9be9-89db8bf3af80-host-slash\") pod \"iptables-alerter-8hwtj\" (UID: \"6c86e38b-1231-4756-9be9-89db8bf3af80\") " pod="openshift-network-operator/iptables-alerter-8hwtj" Sep 10 03:47:55.708738 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.708242 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-multus-conf-dir\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.708738 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.708281 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ddad2c33-abd1-44e0-a8f0-1180786284b8-etc-sysconfig\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.708738 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.708309 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ddad2c33-abd1-44e0-a8f0-1180786284b8-etc-kubernetes\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.708738 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.708466 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ddad2c33-abd1-44e0-a8f0-1180786284b8-etc-kubernetes\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.708962 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.708782 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6c86e38b-1231-4756-9be9-89db8bf3af80-iptables-alerter-script\") pod \"iptables-alerter-8hwtj\" (UID: \"6c86e38b-1231-4756-9be9-89db8bf3af80\") " pod="openshift-network-operator/iptables-alerter-8hwtj" Sep 10 03:47:55.708962 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.708881 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-multus-conf-dir\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.709067 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.708975 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-host-var-lib-cni-multus\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.709067 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.709021 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ddad2c33-abd1-44e0-a8f0-1180786284b8-etc-sysctl-d\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.709067 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.709047 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ddad2c33-abd1-44e0-a8f0-1180786284b8-var-lib-kubelet\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.709234 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.709085 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ddad2c33-abd1-44e0-a8f0-1180786284b8-etc-sysconfig\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.709234 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.709126 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/348bacd3-9294-48e6-bbf4-60727b9a7bfb-dbus\") pod \"global-pull-secret-syncer-gwzk7\" (UID: \"348bacd3-9294-48e6-bbf4-60727b9a7bfb\") " pod="kube-system/global-pull-secret-syncer-gwzk7" Sep 10 03:47:55.709234 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.709161 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-host-run-k8s-cni-cncf-io\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.709234 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.709184 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ddad2c33-abd1-44e0-a8f0-1180786284b8-host\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.709234 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.709198 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ddad2c33-abd1-44e0-a8f0-1180786284b8-etc-sysctl-conf\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.709234 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.709034 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-host-var-lib-kubelet\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.709575 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.709238 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-host-var-lib-cni-bin\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.709575 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.709242 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ddad2c33-abd1-44e0-a8f0-1180786284b8-etc-modprobe-d\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.709575 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.709259 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-multus-cni-dir\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.709575 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.709274 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-multus-socket-dir-parent\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.709575 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.709289 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-cnibin\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.709575 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.709312 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ddad2c33-abd1-44e0-a8f0-1180786284b8-etc-systemd\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.709575 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.709347 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-system-cni-dir\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.709575 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.709403 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-etc-kubernetes\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.709575 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:55.709418 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 03:47:55.709575 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.709454 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-host-run-netns\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.709575 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.709467 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b0936f72-795e-4e46-8316-ecc03bb2dec6-host-run-multus-certs\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.709575 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.709423 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/348bacd3-9294-48e6-bbf4-60727b9a7bfb-kubelet-config\") pod \"global-pull-secret-syncer-gwzk7\" (UID: \"348bacd3-9294-48e6-bbf4-60727b9a7bfb\") " pod="kube-system/global-pull-secret-syncer-gwzk7" Sep 10 03:47:55.709575 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:55.709503 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/348bacd3-9294-48e6-bbf4-60727b9a7bfb-original-pull-secret podName:348bacd3-9294-48e6-bbf4-60727b9a7bfb nodeName:}" failed. No retries permitted until 2026-09-10 03:47:56.209483619 +0000 UTC m=+3.085546751 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/348bacd3-9294-48e6-bbf4-60727b9a7bfb-original-pull-secret") pod "global-pull-secret-syncer-gwzk7" (UID: "348bacd3-9294-48e6-bbf4-60727b9a7bfb") : object "kube-system"/"original-pull-secret" not registered Sep 10 03:47:55.709575 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.709571 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ddad2c33-abd1-44e0-a8f0-1180786284b8-run\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.710151 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.709917 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b0936f72-795e-4e46-8316-ecc03bb2dec6-multus-daemon-config\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.710151 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.710032 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/46180a14-d2d5-44da-8ed7-80a47696fcf7-konnectivity-ca\") pod \"konnectivity-agent-6ds5c\" (UID: \"46180a14-d2d5-44da-8ed7-80a47696fcf7\") " pod="kube-system/konnectivity-agent-6ds5c" Sep 10 03:47:55.711766 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.711669 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/46180a14-d2d5-44da-8ed7-80a47696fcf7-agent-certs\") pod \"konnectivity-agent-6ds5c\" (UID: \"46180a14-d2d5-44da-8ed7-80a47696fcf7\") " pod="kube-system/konnectivity-agent-6ds5c" Sep 10 03:47:55.711895 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.711874 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ddad2c33-abd1-44e0-a8f0-1180786284b8-tmp\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.712462 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.712436 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ddad2c33-abd1-44e0-a8f0-1180786284b8-etc-tuned\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.720063 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:55.719573 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 03:47:55.720063 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:55.719597 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 03:47:55.720063 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:55.719615 2562 projected.go:196] Error preparing data for projected volume kube-api-access-rhrqr for pod openshift-network-diagnostics/network-check-target-hlv2m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 03:47:55.720063 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:55.719701 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10447409-1646-48fb-a5c9-494670b7d232-kube-api-access-rhrqr podName:10447409-1646-48fb-a5c9-494670b7d232 nodeName:}" failed. No retries permitted until 2026-09-10 03:47:56.2196817 +0000 UTC m=+3.095744844 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rhrqr" (UniqueName: "kubernetes.io/projected/10447409-1646-48fb-a5c9-494670b7d232-kube-api-access-rhrqr") pod "network-check-target-hlv2m" (UID: "10447409-1646-48fb-a5c9-494670b7d232") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 03:47:55.720443 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.720345 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv48g\" (UniqueName: \"kubernetes.io/projected/6c86e38b-1231-4756-9be9-89db8bf3af80-kube-api-access-xv48g\") pod \"iptables-alerter-8hwtj\" (UID: \"6c86e38b-1231-4756-9be9-89db8bf3af80\") " pod="openshift-network-operator/iptables-alerter-8hwtj" Sep 10 03:47:55.721355 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.721330 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfn49\" (UniqueName: \"kubernetes.io/projected/b0936f72-795e-4e46-8316-ecc03bb2dec6-kube-api-access-bfn49\") pod \"multus-vwrxj\" (UID: \"b0936f72-795e-4e46-8316-ecc03bb2dec6\") " pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.723064 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.723034 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnln8\" (UniqueName: \"kubernetes.io/projected/ddad2c33-abd1-44e0-a8f0-1180786284b8-kube-api-access-wnln8\") pod \"tuned-8bb28\" (UID: \"ddad2c33-abd1-44e0-a8f0-1180786284b8\") " pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.723150 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.723082 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fx4rm\" (UniqueName: \"kubernetes.io/projected/158bac16-c713-47cd-91d7-9fc36d96f570-kube-api-access-fx4rm\") pod \"network-metrics-daemon-2p9b8\" (UID: \"158bac16-c713-47cd-91d7-9fc36d96f570\") " pod="openshift-multus/network-metrics-daemon-2p9b8" Sep 10 03:47:55.792394 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.792344 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ckqrf" Sep 10 03:47:55.798384 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.798342 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkxrh" Sep 10 03:47:55.807355 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.807296 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:47:55.813586 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.813376 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fwlqw" Sep 10 03:47:55.818834 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:55.815902 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaae7f13d_be0c_4bd9_af61_40fc05f144c0.slice/crio-d7e70bc1ef0ab2640e2915740aaf2026f25c174b12b0b13c5fc55bc9e92e0499 WatchSource:0}: Error finding container d7e70bc1ef0ab2640e2915740aaf2026f25c174b12b0b13c5fc55bc9e92e0499: Status 404 returned error can't find the container with id d7e70bc1ef0ab2640e2915740aaf2026f25c174b12b0b13c5fc55bc9e92e0499 Sep 10 03:47:55.818834 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.818823 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nmf2n" Sep 10 03:47:55.826209 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.826000 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8bb28" Sep 10 03:47:55.834063 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.832665 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8hwtj" Sep 10 03:47:55.834063 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:55.833039 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod460613b7_2cdf_4e57_8edf_06e7b72f790a.slice/crio-1ddf3f862a2e5e345596d9d22fc8f69b9205566cd696f109f1ee064a09313787 WatchSource:0}: Error finding container 1ddf3f862a2e5e345596d9d22fc8f69b9205566cd696f109f1ee064a09313787: Status 404 returned error can't find the container with id 1ddf3f862a2e5e345596d9d22fc8f69b9205566cd696f109f1ee064a09313787 Sep 10 03:47:55.838110 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.838085 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6ds5c" Sep 10 03:47:55.841216 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:55.841172 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ef729ce_06d8_4c1d_ac8c_7a89770df17f.slice/crio-e2cb731de924995ea055fb073853c053a2dbe5d4db56b5d5d20f1df4be559753 WatchSource:0}: Error finding container e2cb731de924995ea055fb073853c053a2dbe5d4db56b5d5d20f1df4be559753: Status 404 returned error can't find the container with id e2cb731de924995ea055fb073853c053a2dbe5d4db56b5d5d20f1df4be559753 Sep 10 03:47:55.842968 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:55.842795 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vwrxj" Sep 10 03:47:55.849887 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:55.849840 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddad2c33_abd1_44e0_a8f0_1180786284b8.slice/crio-8844911f66c5a7b040e33027606ff6b74b716871bb094052bbadc91dd681eb3f WatchSource:0}: Error finding container 8844911f66c5a7b040e33027606ff6b74b716871bb094052bbadc91dd681eb3f: Status 404 returned error can't find the container with id 8844911f66c5a7b040e33027606ff6b74b716871bb094052bbadc91dd681eb3f Sep 10 03:47:55.854696 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:47:55.854664 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c86e38b_1231_4756_9be9_89db8bf3af80.slice/crio-a4663d42971650f4e17cf5ab7ea2645e308d08410cb29f6be6bb9b7c89bed252 WatchSource:0}: Error finding container a4663d42971650f4e17cf5ab7ea2645e308d08410cb29f6be6bb9b7c89bed252: Status 404 returned error can't find the container with id a4663d42971650f4e17cf5ab7ea2645e308d08410cb29f6be6bb9b7c89bed252 Sep 10 03:47:56.212098 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:56.212011 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/348bacd3-9294-48e6-bbf4-60727b9a7bfb-original-pull-secret\") pod \"global-pull-secret-syncer-gwzk7\" (UID: \"348bacd3-9294-48e6-bbf4-60727b9a7bfb\") " pod="kube-system/global-pull-secret-syncer-gwzk7" Sep 10 03:47:56.212098 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:56.212084 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158bac16-c713-47cd-91d7-9fc36d96f570-metrics-certs\") pod \"network-metrics-daemon-2p9b8\" (UID: \"158bac16-c713-47cd-91d7-9fc36d96f570\") " pod="openshift-multus/network-metrics-daemon-2p9b8" Sep 10 03:47:56.212304 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:56.212197 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 03:47:56.212304 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:56.212285 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/348bacd3-9294-48e6-bbf4-60727b9a7bfb-original-pull-secret podName:348bacd3-9294-48e6-bbf4-60727b9a7bfb nodeName:}" failed. No retries permitted until 2026-09-10 03:47:57.212264018 +0000 UTC m=+4.088327155 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/348bacd3-9294-48e6-bbf4-60727b9a7bfb-original-pull-secret") pod "global-pull-secret-syncer-gwzk7" (UID: "348bacd3-9294-48e6-bbf4-60727b9a7bfb") : object "kube-system"/"original-pull-secret" not registered Sep 10 03:47:56.212304 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:56.212207 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 03:47:56.212492 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:56.212357 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/158bac16-c713-47cd-91d7-9fc36d96f570-metrics-certs podName:158bac16-c713-47cd-91d7-9fc36d96f570 nodeName:}" failed. No retries permitted until 2026-09-10 03:47:57.212344362 +0000 UTC m=+4.088407495 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/158bac16-c713-47cd-91d7-9fc36d96f570-metrics-certs") pod "network-metrics-daemon-2p9b8" (UID: "158bac16-c713-47cd-91d7-9fc36d96f570") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 03:47:56.312683 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:56.312638 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhrqr\" (UniqueName: \"kubernetes.io/projected/10447409-1646-48fb-a5c9-494670b7d232-kube-api-access-rhrqr\") pod \"network-check-target-hlv2m\" (UID: \"10447409-1646-48fb-a5c9-494670b7d232\") " pod="openshift-network-diagnostics/network-check-target-hlv2m" Sep 10 03:47:56.312892 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:56.312823 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 03:47:56.312892 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:56.312845 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 03:47:56.312892 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:56.312855 2562 projected.go:196] Error preparing data for projected volume kube-api-access-rhrqr for pod openshift-network-diagnostics/network-check-target-hlv2m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 03:47:56.313069 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:56.312918 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10447409-1646-48fb-a5c9-494670b7d232-kube-api-access-rhrqr podName:10447409-1646-48fb-a5c9-494670b7d232 nodeName:}" failed. No retries permitted until 2026-09-10 03:47:57.312904127 +0000 UTC m=+4.188967264 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rhrqr" (UniqueName: "kubernetes.io/projected/10447409-1646-48fb-a5c9-494670b7d232-kube-api-access-rhrqr") pod "network-check-target-hlv2m" (UID: "10447409-1646-48fb-a5c9-494670b7d232") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 03:47:56.348398 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:56.348364 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 03:47:56.539476 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:56.539370 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 03:42:55 +0000 UTC" deadline="2028-03-28 05:51:37.100943869 +0000 UTC" Sep 10 03:47:56.539476 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:56.539432 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13562h3m40.561517761s" Sep 10 03:47:56.608950 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:56.608907 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8hwtj" event={"ID":"6c86e38b-1231-4756-9be9-89db8bf3af80","Type":"ContainerStarted","Data":"a4663d42971650f4e17cf5ab7ea2645e308d08410cb29f6be6bb9b7c89bed252"} Sep 10 03:47:56.610371 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:56.610338 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8bb28" event={"ID":"ddad2c33-abd1-44e0-a8f0-1180786284b8","Type":"ContainerStarted","Data":"8844911f66c5a7b040e33027606ff6b74b716871bb094052bbadc91dd681eb3f"} Sep 10 03:47:56.611769 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:56.611742 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nmf2n" event={"ID":"1ef729ce-06d8-4c1d-ac8c-7a89770df17f","Type":"ContainerStarted","Data":"e2cb731de924995ea055fb073853c053a2dbe5d4db56b5d5d20f1df4be559753"} Sep 10 03:47:56.612993 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:56.612956 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fwlqw" event={"ID":"460613b7-2cdf-4e57-8edf-06e7b72f790a","Type":"ContainerStarted","Data":"1ddf3f862a2e5e345596d9d22fc8f69b9205566cd696f109f1ee064a09313787"} Sep 10 03:47:56.614405 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:56.614379 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" event={"ID":"8459a996-8d64-4c23-9e92-1a5bcca253b5","Type":"ContainerStarted","Data":"02ac5cb227a08d33b27a147d31cff8e0eab13652740bd5272877b2911313b20d"} Sep 10 03:47:56.615711 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:56.615688 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkxrh" event={"ID":"434f994a-63b7-4290-a72a-eb55456f4790","Type":"ContainerStarted","Data":"e6f5a278f130a069173d51f8ddcfc087294962babdb703354ea75348e5286c1e"} Sep 10 03:47:56.616994 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:56.616967 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ckqrf" event={"ID":"aae7f13d-be0c-4bd9-af61-40fc05f144c0","Type":"ContainerStarted","Data":"d7e70bc1ef0ab2640e2915740aaf2026f25c174b12b0b13c5fc55bc9e92e0499"} Sep 10 03:47:56.618281 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:56.618260 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vwrxj" event={"ID":"b0936f72-795e-4e46-8316-ecc03bb2dec6","Type":"ContainerStarted","Data":"2a55b9488fe4df3dc5ee76bc79292bbfe17b746511e88c38ab24118491963393"} Sep 10 03:47:56.619431 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:56.619409 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6ds5c" event={"ID":"46180a14-d2d5-44da-8ed7-80a47696fcf7","Type":"ContainerStarted","Data":"3036816e1694c93dac1183e33760a7909f525fb3ecaeb44c23b3c80dd9ec6a39"} Sep 10 03:47:56.743631 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:56.743594 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 03:47:57.218776 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:57.218731 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/348bacd3-9294-48e6-bbf4-60727b9a7bfb-original-pull-secret\") pod \"global-pull-secret-syncer-gwzk7\" (UID: \"348bacd3-9294-48e6-bbf4-60727b9a7bfb\") " pod="kube-system/global-pull-secret-syncer-gwzk7" Sep 10 03:47:57.218996 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:57.218826 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158bac16-c713-47cd-91d7-9fc36d96f570-metrics-certs\") pod \"network-metrics-daemon-2p9b8\" (UID: \"158bac16-c713-47cd-91d7-9fc36d96f570\") " pod="openshift-multus/network-metrics-daemon-2p9b8" Sep 10 03:47:57.219084 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:57.219058 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 03:47:57.219157 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:57.219135 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/158bac16-c713-47cd-91d7-9fc36d96f570-metrics-certs podName:158bac16-c713-47cd-91d7-9fc36d96f570 nodeName:}" failed. No retries permitted until 2026-09-10 03:47:59.219113821 +0000 UTC m=+6.095176966 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/158bac16-c713-47cd-91d7-9fc36d96f570-metrics-certs") pod "network-metrics-daemon-2p9b8" (UID: "158bac16-c713-47cd-91d7-9fc36d96f570") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 03:47:57.219879 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:57.219612 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 03:47:57.219879 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:57.219679 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/348bacd3-9294-48e6-bbf4-60727b9a7bfb-original-pull-secret podName:348bacd3-9294-48e6-bbf4-60727b9a7bfb nodeName:}" failed. No retries permitted until 2026-09-10 03:47:59.219659033 +0000 UTC m=+6.095722167 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/348bacd3-9294-48e6-bbf4-60727b9a7bfb-original-pull-secret") pod "global-pull-secret-syncer-gwzk7" (UID: "348bacd3-9294-48e6-bbf4-60727b9a7bfb") : object "kube-system"/"original-pull-secret" not registered Sep 10 03:47:57.320506 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:57.319700 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhrqr\" (UniqueName: \"kubernetes.io/projected/10447409-1646-48fb-a5c9-494670b7d232-kube-api-access-rhrqr\") pod \"network-check-target-hlv2m\" (UID: \"10447409-1646-48fb-a5c9-494670b7d232\") " pod="openshift-network-diagnostics/network-check-target-hlv2m" Sep 10 03:47:57.320506 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:57.319909 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 03:47:57.320506 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:57.319935 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 03:47:57.320506 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:57.319951 2562 projected.go:196] Error preparing data for projected volume kube-api-access-rhrqr for pod openshift-network-diagnostics/network-check-target-hlv2m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 03:47:57.320506 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:57.320023 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10447409-1646-48fb-a5c9-494670b7d232-kube-api-access-rhrqr podName:10447409-1646-48fb-a5c9-494670b7d232 nodeName:}" failed. No retries permitted until 2026-09-10 03:47:59.319999569 +0000 UTC m=+6.196062703 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rhrqr" (UniqueName: "kubernetes.io/projected/10447409-1646-48fb-a5c9-494670b7d232-kube-api-access-rhrqr") pod "network-check-target-hlv2m" (UID: "10447409-1646-48fb-a5c9-494670b7d232") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 03:47:57.540048 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:57.539916 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 03:42:55 +0000 UTC" deadline="2028-03-02 04:06:08.738394067 +0000 UTC" Sep 10 03:47:57.540048 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:57.539975 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12936h18m11.198423595s" Sep 10 03:47:57.598470 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:57.598437 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2p9b8" Sep 10 03:47:57.598653 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:57.598586 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2p9b8" podUID="158bac16-c713-47cd-91d7-9fc36d96f570" Sep 10 03:47:57.599013 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:57.598989 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gwzk7" Sep 10 03:47:57.599117 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:57.599096 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gwzk7" podUID="348bacd3-9294-48e6-bbf4-60727b9a7bfb" Sep 10 03:47:57.599263 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:57.599241 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hlv2m" Sep 10 03:47:57.599363 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:57.599339 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hlv2m" podUID="10447409-1646-48fb-a5c9-494670b7d232" Sep 10 03:47:58.641600 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:58.640224 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-74.ec2.internal" event={"ID":"95b230e05074eeabdbb71cf81104af8d","Type":"ContainerStarted","Data":"d369da31cf432f7c0351ad855c7e9868e6eef44a1d9694dbec2fe200a5deb12e"} Sep 10 03:47:59.234505 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:59.234401 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/348bacd3-9294-48e6-bbf4-60727b9a7bfb-original-pull-secret\") pod \"global-pull-secret-syncer-gwzk7\" (UID: \"348bacd3-9294-48e6-bbf4-60727b9a7bfb\") " pod="kube-system/global-pull-secret-syncer-gwzk7" Sep 10 03:47:59.234505 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:59.234489 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158bac16-c713-47cd-91d7-9fc36d96f570-metrics-certs\") pod \"network-metrics-daemon-2p9b8\" (UID: \"158bac16-c713-47cd-91d7-9fc36d96f570\") " pod="openshift-multus/network-metrics-daemon-2p9b8" Sep 10 03:47:59.234837 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:59.234637 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 03:47:59.234837 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:59.234704 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/158bac16-c713-47cd-91d7-9fc36d96f570-metrics-certs podName:158bac16-c713-47cd-91d7-9fc36d96f570 nodeName:}" failed. No retries permitted until 2026-09-10 03:48:03.234683023 +0000 UTC m=+10.110746157 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/158bac16-c713-47cd-91d7-9fc36d96f570-metrics-certs") pod "network-metrics-daemon-2p9b8" (UID: "158bac16-c713-47cd-91d7-9fc36d96f570") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 03:47:59.235116 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:59.235094 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 03:47:59.235207 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:59.235173 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/348bacd3-9294-48e6-bbf4-60727b9a7bfb-original-pull-secret podName:348bacd3-9294-48e6-bbf4-60727b9a7bfb nodeName:}" failed. No retries permitted until 2026-09-10 03:48:03.235137481 +0000 UTC m=+10.111200617 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/348bacd3-9294-48e6-bbf4-60727b9a7bfb-original-pull-secret") pod "global-pull-secret-syncer-gwzk7" (UID: "348bacd3-9294-48e6-bbf4-60727b9a7bfb") : object "kube-system"/"original-pull-secret" not registered Sep 10 03:47:59.335804 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:59.335118 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhrqr\" (UniqueName: \"kubernetes.io/projected/10447409-1646-48fb-a5c9-494670b7d232-kube-api-access-rhrqr\") pod \"network-check-target-hlv2m\" (UID: \"10447409-1646-48fb-a5c9-494670b7d232\") " pod="openshift-network-diagnostics/network-check-target-hlv2m" Sep 10 03:47:59.335804 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:59.335359 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 03:47:59.335804 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:59.335382 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 03:47:59.335804 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:59.335395 2562 projected.go:196] Error preparing data for projected volume kube-api-access-rhrqr for pod openshift-network-diagnostics/network-check-target-hlv2m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 03:47:59.335804 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:59.335452 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10447409-1646-48fb-a5c9-494670b7d232-kube-api-access-rhrqr podName:10447409-1646-48fb-a5c9-494670b7d232 nodeName:}" failed. No retries permitted until 2026-09-10 03:48:03.335433744 +0000 UTC m=+10.211496876 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rhrqr" (UniqueName: "kubernetes.io/projected/10447409-1646-48fb-a5c9-494670b7d232-kube-api-access-rhrqr") pod "network-check-target-hlv2m" (UID: "10447409-1646-48fb-a5c9-494670b7d232") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 03:47:59.599984 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:59.599400 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2p9b8" Sep 10 03:47:59.599984 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:59.599551 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2p9b8" podUID="158bac16-c713-47cd-91d7-9fc36d96f570" Sep 10 03:47:59.600389 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:59.600358 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hlv2m" Sep 10 03:47:59.600497 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:59.600460 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hlv2m" podUID="10447409-1646-48fb-a5c9-494670b7d232" Sep 10 03:47:59.600560 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:59.600529 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gwzk7" Sep 10 03:47:59.600622 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:47:59.600603 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gwzk7" podUID="348bacd3-9294-48e6-bbf4-60727b9a7bfb" Sep 10 03:47:59.656181 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:59.656144 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.ec2.internal" event={"ID":"d8b7ce27f94ad4477741ec9d86dc3b51","Type":"ContainerStarted","Data":"3262abce2239e105ced360d0c95153618b8331ba5c1a1937b6bb305bf5af8537"} Sep 10 03:47:59.679268 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:47:59.679200 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-74.ec2.internal" podStartSLOduration=4.67917685 podStartE2EDuration="4.67917685s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 03:47:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:47:58.656025996 +0000 UTC m=+5.532089148" watchObservedRunningTime="2026-09-10 03:47:59.67917685 +0000 UTC m=+6.555240003" Sep 10 03:48:01.599026 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:01.598992 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2p9b8" Sep 10 03:48:01.599510 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:01.599000 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hlv2m" Sep 10 03:48:01.599510 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:01.599145 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2p9b8" podUID="158bac16-c713-47cd-91d7-9fc36d96f570" Sep 10 03:48:01.599510 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:01.599230 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hlv2m" podUID="10447409-1646-48fb-a5c9-494670b7d232" Sep 10 03:48:01.599510 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:01.599000 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gwzk7" Sep 10 03:48:01.599510 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:01.599344 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gwzk7" podUID="348bacd3-9294-48e6-bbf4-60727b9a7bfb" Sep 10 03:48:01.663393 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:01.663354 2562 generic.go:356] "Generic (PLEG): container finished" podID="d8b7ce27f94ad4477741ec9d86dc3b51" containerID="3262abce2239e105ced360d0c95153618b8331ba5c1a1937b6bb305bf5af8537" exitCode=0 Sep 10 03:48:01.663568 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:01.663405 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.ec2.internal" event={"ID":"d8b7ce27f94ad4477741ec9d86dc3b51","Type":"ContainerDied","Data":"3262abce2239e105ced360d0c95153618b8331ba5c1a1937b6bb305bf5af8537"} Sep 10 03:48:03.267628 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:03.266786 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/348bacd3-9294-48e6-bbf4-60727b9a7bfb-original-pull-secret\") pod \"global-pull-secret-syncer-gwzk7\" (UID: \"348bacd3-9294-48e6-bbf4-60727b9a7bfb\") " pod="kube-system/global-pull-secret-syncer-gwzk7" Sep 10 03:48:03.267628 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:03.266864 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158bac16-c713-47cd-91d7-9fc36d96f570-metrics-certs\") pod \"network-metrics-daemon-2p9b8\" (UID: \"158bac16-c713-47cd-91d7-9fc36d96f570\") " pod="openshift-multus/network-metrics-daemon-2p9b8" Sep 10 03:48:03.267628 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:03.266988 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 03:48:03.267628 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:03.267054 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/158bac16-c713-47cd-91d7-9fc36d96f570-metrics-certs podName:158bac16-c713-47cd-91d7-9fc36d96f570 nodeName:}" failed. No retries permitted until 2026-09-10 03:48:11.267035072 +0000 UTC m=+18.143098205 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/158bac16-c713-47cd-91d7-9fc36d96f570-metrics-certs") pod "network-metrics-daemon-2p9b8" (UID: "158bac16-c713-47cd-91d7-9fc36d96f570") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 03:48:03.267628 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:03.267526 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 03:48:03.267628 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:03.267594 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/348bacd3-9294-48e6-bbf4-60727b9a7bfb-original-pull-secret podName:348bacd3-9294-48e6-bbf4-60727b9a7bfb nodeName:}" failed. No retries permitted until 2026-09-10 03:48:11.267578295 +0000 UTC m=+18.143641430 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/348bacd3-9294-48e6-bbf4-60727b9a7bfb-original-pull-secret") pod "global-pull-secret-syncer-gwzk7" (UID: "348bacd3-9294-48e6-bbf4-60727b9a7bfb") : object "kube-system"/"original-pull-secret" not registered Sep 10 03:48:03.368067 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:03.367607 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhrqr\" (UniqueName: \"kubernetes.io/projected/10447409-1646-48fb-a5c9-494670b7d232-kube-api-access-rhrqr\") pod \"network-check-target-hlv2m\" (UID: \"10447409-1646-48fb-a5c9-494670b7d232\") " pod="openshift-network-diagnostics/network-check-target-hlv2m" Sep 10 03:48:03.368067 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:03.367843 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 03:48:03.368067 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:03.367864 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 03:48:03.368067 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:03.367878 2562 projected.go:196] Error preparing data for projected volume kube-api-access-rhrqr for pod openshift-network-diagnostics/network-check-target-hlv2m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 03:48:03.368067 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:03.367939 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10447409-1646-48fb-a5c9-494670b7d232-kube-api-access-rhrqr podName:10447409-1646-48fb-a5c9-494670b7d232 nodeName:}" failed. No retries permitted until 2026-09-10 03:48:11.367920995 +0000 UTC m=+18.243984129 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rhrqr" (UniqueName: "kubernetes.io/projected/10447409-1646-48fb-a5c9-494670b7d232-kube-api-access-rhrqr") pod "network-check-target-hlv2m" (UID: "10447409-1646-48fb-a5c9-494670b7d232") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 03:48:03.600444 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:03.600366 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2p9b8" Sep 10 03:48:03.600610 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:03.600499 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2p9b8" podUID="158bac16-c713-47cd-91d7-9fc36d96f570" Sep 10 03:48:03.600907 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:03.600884 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gwzk7" Sep 10 03:48:03.601012 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:03.600991 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gwzk7" podUID="348bacd3-9294-48e6-bbf4-60727b9a7bfb" Sep 10 03:48:03.601079 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:03.601043 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hlv2m" Sep 10 03:48:03.601129 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:03.601110 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hlv2m" podUID="10447409-1646-48fb-a5c9-494670b7d232" Sep 10 03:48:05.598690 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:05.598658 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2p9b8" Sep 10 03:48:05.599144 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:05.598705 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hlv2m" Sep 10 03:48:05.599144 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:05.598658 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gwzk7" Sep 10 03:48:05.599144 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:05.598799 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2p9b8" podUID="158bac16-c713-47cd-91d7-9fc36d96f570" Sep 10 03:48:05.599144 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:05.598873 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gwzk7" podUID="348bacd3-9294-48e6-bbf4-60727b9a7bfb" Sep 10 03:48:05.599144 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:05.598987 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hlv2m" podUID="10447409-1646-48fb-a5c9-494670b7d232" Sep 10 03:48:07.599031 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:07.598993 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2p9b8" Sep 10 03:48:07.599504 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:07.598993 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hlv2m" Sep 10 03:48:07.599504 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:07.599130 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2p9b8" podUID="158bac16-c713-47cd-91d7-9fc36d96f570" Sep 10 03:48:07.599504 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:07.598996 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gwzk7" Sep 10 03:48:07.599504 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:07.599228 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hlv2m" podUID="10447409-1646-48fb-a5c9-494670b7d232" Sep 10 03:48:07.599504 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:07.599269 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gwzk7" podUID="348bacd3-9294-48e6-bbf4-60727b9a7bfb" Sep 10 03:48:09.598833 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:09.598789 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gwzk7" Sep 10 03:48:09.598833 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:09.598818 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2p9b8" Sep 10 03:48:09.599376 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:09.598790 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hlv2m" Sep 10 03:48:09.599376 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:09.598926 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gwzk7" podUID="348bacd3-9294-48e6-bbf4-60727b9a7bfb" Sep 10 03:48:09.599376 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:09.599001 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hlv2m" podUID="10447409-1646-48fb-a5c9-494670b7d232" Sep 10 03:48:09.599376 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:09.599093 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2p9b8" podUID="158bac16-c713-47cd-91d7-9fc36d96f570" Sep 10 03:48:11.329207 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:11.329165 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/348bacd3-9294-48e6-bbf4-60727b9a7bfb-original-pull-secret\") pod \"global-pull-secret-syncer-gwzk7\" (UID: \"348bacd3-9294-48e6-bbf4-60727b9a7bfb\") " pod="kube-system/global-pull-secret-syncer-gwzk7" Sep 10 03:48:11.329751 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:11.329229 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158bac16-c713-47cd-91d7-9fc36d96f570-metrics-certs\") pod \"network-metrics-daemon-2p9b8\" (UID: \"158bac16-c713-47cd-91d7-9fc36d96f570\") " pod="openshift-multus/network-metrics-daemon-2p9b8" Sep 10 03:48:11.329751 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:11.329359 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 03:48:11.329751 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:11.329444 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/348bacd3-9294-48e6-bbf4-60727b9a7bfb-original-pull-secret podName:348bacd3-9294-48e6-bbf4-60727b9a7bfb nodeName:}" failed. No retries permitted until 2026-09-10 03:48:27.329418956 +0000 UTC m=+34.205482088 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/348bacd3-9294-48e6-bbf4-60727b9a7bfb-original-pull-secret") pod "global-pull-secret-syncer-gwzk7" (UID: "348bacd3-9294-48e6-bbf4-60727b9a7bfb") : object "kube-system"/"original-pull-secret" not registered Sep 10 03:48:11.329751 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:11.329360 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 03:48:11.329751 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:11.329509 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/158bac16-c713-47cd-91d7-9fc36d96f570-metrics-certs podName:158bac16-c713-47cd-91d7-9fc36d96f570 nodeName:}" failed. No retries permitted until 2026-09-10 03:48:27.329495364 +0000 UTC m=+34.205558508 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/158bac16-c713-47cd-91d7-9fc36d96f570-metrics-certs") pod "network-metrics-daemon-2p9b8" (UID: "158bac16-c713-47cd-91d7-9fc36d96f570") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 03:48:11.430529 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:11.430484 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhrqr\" (UniqueName: \"kubernetes.io/projected/10447409-1646-48fb-a5c9-494670b7d232-kube-api-access-rhrqr\") pod \"network-check-target-hlv2m\" (UID: \"10447409-1646-48fb-a5c9-494670b7d232\") " pod="openshift-network-diagnostics/network-check-target-hlv2m" Sep 10 03:48:11.430712 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:11.430669 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 03:48:11.430712 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:11.430692 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 03:48:11.430712 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:11.430702 2562 projected.go:196] Error preparing data for projected volume kube-api-access-rhrqr for pod openshift-network-diagnostics/network-check-target-hlv2m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 03:48:11.430845 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:11.430759 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10447409-1646-48fb-a5c9-494670b7d232-kube-api-access-rhrqr podName:10447409-1646-48fb-a5c9-494670b7d232 nodeName:}" failed. No retries permitted until 2026-09-10 03:48:27.430745774 +0000 UTC m=+34.306808908 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rhrqr" (UniqueName: "kubernetes.io/projected/10447409-1646-48fb-a5c9-494670b7d232-kube-api-access-rhrqr") pod "network-check-target-hlv2m" (UID: "10447409-1646-48fb-a5c9-494670b7d232") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 03:48:11.599209 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:11.599120 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hlv2m" Sep 10 03:48:11.599391 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:11.599120 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2p9b8" Sep 10 03:48:11.599391 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:11.599247 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hlv2m" podUID="10447409-1646-48fb-a5c9-494670b7d232" Sep 10 03:48:11.599391 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:11.599258 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gwzk7" Sep 10 03:48:11.599391 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:11.599376 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2p9b8" podUID="158bac16-c713-47cd-91d7-9fc36d96f570" Sep 10 03:48:11.599606 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:11.599469 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gwzk7" podUID="348bacd3-9294-48e6-bbf4-60727b9a7bfb" Sep 10 03:48:13.600213 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:13.600171 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2p9b8" Sep 10 03:48:13.600680 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:13.600447 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2p9b8" podUID="158bac16-c713-47cd-91d7-9fc36d96f570" Sep 10 03:48:13.600680 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:13.600468 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hlv2m" Sep 10 03:48:13.600800 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:13.600694 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hlv2m" podUID="10447409-1646-48fb-a5c9-494670b7d232" Sep 10 03:48:13.600800 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:13.600774 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gwzk7" Sep 10 03:48:13.600982 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:13.600961 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gwzk7" podUID="348bacd3-9294-48e6-bbf4-60727b9a7bfb" Sep 10 03:48:14.691599 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:14.691333 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nmf2n" event={"ID":"1ef729ce-06d8-4c1d-ac8c-7a89770df17f","Type":"ContainerStarted","Data":"ec5d277a08486599852e7bc20eeb43004bb6fe4518c8c86d2be6822abf0c7ac0"} Sep 10 03:48:14.694961 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:14.694931 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fwlqw" event={"ID":"460613b7-2cdf-4e57-8edf-06e7b72f790a","Type":"ContainerStarted","Data":"f87eb3cee51d5a02cde4c732fae3a6d8f43f382a39c537a7a2f4f0c407d8d664"} Sep 10 03:48:14.697201 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:14.697175 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fl7hj_8459a996-8d64-4c23-9e92-1a5bcca253b5/ovn-acl-logging/0.log" Sep 10 03:48:14.697563 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:14.697538 2562 generic.go:356] "Generic (PLEG): container finished" podID="8459a996-8d64-4c23-9e92-1a5bcca253b5" containerID="fe31bd49803112d5d592fad1d17dd5506bf8a9b63bf7dbe67bf7b0d06253d2ff" exitCode=1 Sep 10 03:48:14.697682 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:14.697610 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" event={"ID":"8459a996-8d64-4c23-9e92-1a5bcca253b5","Type":"ContainerStarted","Data":"bc0d92d7de8dfe6ebf0c3402b17ca21deb153c0228e42942221d085a2e671434"} Sep 10 03:48:14.697744 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:14.697683 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" event={"ID":"8459a996-8d64-4c23-9e92-1a5bcca253b5","Type":"ContainerDied","Data":"fe31bd49803112d5d592fad1d17dd5506bf8a9b63bf7dbe67bf7b0d06253d2ff"} Sep 10 03:48:14.697744 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:14.697717 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" event={"ID":"8459a996-8d64-4c23-9e92-1a5bcca253b5","Type":"ContainerStarted","Data":"86eb9930f6f630acce5bfeff8bbdc40d82e5569140a3423ee2eecd7b6f681ad4"} Sep 10 03:48:14.702376 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:14.702336 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkxrh" event={"ID":"434f994a-63b7-4290-a72a-eb55456f4790","Type":"ContainerStarted","Data":"29424addc5002f51d3d08ad3734c72bc7f661c795472dd76861e8d880f308d63"} Sep 10 03:48:14.703972 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:14.703936 2562 generic.go:356] "Generic (PLEG): container finished" podID="aae7f13d-be0c-4bd9-af61-40fc05f144c0" containerID="69ef43f596067cd7958ee89a43a21a754f25e67bf9b1b9586ff581e0a3615549" exitCode=0 Sep 10 03:48:14.704070 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:14.704007 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ckqrf" event={"ID":"aae7f13d-be0c-4bd9-af61-40fc05f144c0","Type":"ContainerDied","Data":"69ef43f596067cd7958ee89a43a21a754f25e67bf9b1b9586ff581e0a3615549"} Sep 10 03:48:14.706195 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:14.706167 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.ec2.internal" event={"ID":"d8b7ce27f94ad4477741ec9d86dc3b51","Type":"ContainerStarted","Data":"76fe49ada7ae7d86fbafcf9d7c4fe4d42b5a53c372bba5ab1ddf2aa6c241adf2"} Sep 10 03:48:14.707922 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:14.707883 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vwrxj" event={"ID":"b0936f72-795e-4e46-8316-ecc03bb2dec6","Type":"ContainerStarted","Data":"5d696b579713c0e1ce43a09ed0924c72a026cec6fdef32e3358add6e6d6778b0"} Sep 10 03:48:14.708016 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:14.707954 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-nmf2n" podStartSLOduration=2.338619482 podStartE2EDuration="20.707936729s" totalImagesPullingTime="18.369317247s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 03:47:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:48:14.707154853 +0000 UTC m=+21.583218032" watchObservedRunningTime="2026-09-10 03:48:14.707936729 +0000 UTC m=+21.583999882" Sep 10 03:48:14.709620 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:14.709589 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6ds5c" event={"ID":"46180a14-d2d5-44da-8ed7-80a47696fcf7","Type":"ContainerStarted","Data":"86acec3acd934f751aa45655043e3779e7fbf726e0957edfbd969d8fff08a3bb"} Sep 10 03:48:14.711262 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:14.711239 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8bb28" event={"ID":"ddad2c33-abd1-44e0-a8f0-1180786284b8","Type":"ContainerStarted","Data":"9c8196261252fcfd1e3ce15ed2fd7ef61fbb10ebc462b852a4fdf9c8c2b61d2b"} Sep 10 03:48:14.739877 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:14.739815 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-fwlqw" podStartSLOduration=2.360598737 podStartE2EDuration="20.739791706s" totalImagesPullingTime="18.379192969s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 03:47:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:48:14.738719035 +0000 UTC m=+21.614782187" watchObservedRunningTime="2026-09-10 03:48:14.739791706 +0000 UTC m=+21.615854859" Sep 10 03:48:14.752171 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:14.752122 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.ec2.internal" podStartSLOduration=19.75210605 podStartE2EDuration="19.75210605s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 03:47:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:48:14.751784299 +0000 UTC m=+21.627847451" watchObservedRunningTime="2026-09-10 03:48:14.75210605 +0000 UTC m=+21.628169201" Sep 10 03:48:14.767907 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:14.767634 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8bb28" podStartSLOduration=2.403307361 podStartE2EDuration="20.767612403s" totalImagesPullingTime="18.364305042s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 03:47:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:48:14.767364987 +0000 UTC m=+21.643428140" watchObservedRunningTime="2026-09-10 03:48:14.767612403 +0000 UTC m=+21.643675556" Sep 10 03:48:14.779707 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:14.779659 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6ds5c" podStartSLOduration=2.43950284 podStartE2EDuration="20.779643295s" totalImagesPullingTime="18.340140455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:47:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:48:14.779580094 +0000 UTC m=+21.655643246" watchObservedRunningTime="2026-09-10 03:48:14.779643295 +0000 UTC m=+21.655706447" Sep 10 03:48:14.799117 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:14.799051 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-vwrxj" podStartSLOduration=2.122045344 podStartE2EDuration="20.799034433s" totalImagesPullingTime="18.676989089s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 03:47:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:48:14.798891368 +0000 UTC m=+21.674954521" watchObservedRunningTime="2026-09-10 03:48:14.799034433 +0000 UTC m=+21.675097584" Sep 10 03:48:15.098879 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:15.098834 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6ds5c" Sep 10 03:48:15.099830 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:15.099769 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6ds5c" Sep 10 03:48:15.568252 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:15.568226 2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 10 03:48:15.598591 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:15.598555 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gwzk7" Sep 10 03:48:15.598755 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:15.598555 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2p9b8" Sep 10 03:48:15.598755 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:15.598674 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gwzk7" podUID="348bacd3-9294-48e6-bbf4-60727b9a7bfb" Sep 10 03:48:15.598755 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:15.598555 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hlv2m" Sep 10 03:48:15.598874 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:15.598739 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2p9b8" podUID="158bac16-c713-47cd-91d7-9fc36d96f570" Sep 10 03:48:15.598874 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:15.598847 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hlv2m" podUID="10447409-1646-48fb-a5c9-494670b7d232" Sep 10 03:48:15.716364 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:15.716193 2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T03:48:15.568246828Z","UUID":"9b36f2d8-034e-417e-b1dc-5d08ed8bd18d","Handler":null,"Name":"","Endpoint":""} Sep 10 03:48:15.717057 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:15.717032 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fl7hj_8459a996-8d64-4c23-9e92-1a5bcca253b5/ovn-acl-logging/0.log" Sep 10 03:48:15.717998 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:15.717841 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" event={"ID":"8459a996-8d64-4c23-9e92-1a5bcca253b5","Type":"ContainerStarted","Data":"341f85af1b8707036f587b8940aa6761832193966865ffbc6e71b9a589227a63"} Sep 10 03:48:15.717998 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:15.717927 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" event={"ID":"8459a996-8d64-4c23-9e92-1a5bcca253b5","Type":"ContainerStarted","Data":"330076cec4249aad67f1af46b02f8e188c1ee05cb2ee8b81feddd9994ac1cbcc"} Sep 10 03:48:15.717998 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:15.717975 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" event={"ID":"8459a996-8d64-4c23-9e92-1a5bcca253b5","Type":"ContainerStarted","Data":"2a3b013e66ab2aa2c2dc5cb311081c4243d6b9e94666a1202121e3d63c72f0c0"} Sep 10 03:48:15.719417 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:15.719395 2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 10 03:48:15.719527 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:15.719434 2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 10 03:48:15.723480 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:15.723444 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkxrh" event={"ID":"434f994a-63b7-4290-a72a-eb55456f4790","Type":"ContainerStarted","Data":"b8c417c88485e3cf87301baaaa7079ecd512553e016dc03cd82aba2549a370f5"} Sep 10 03:48:16.725998 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:16.725957 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8hwtj" event={"ID":"6c86e38b-1231-4756-9be9-89db8bf3af80","Type":"ContainerStarted","Data":"08375cd081a92230d0a156f1ca57739ed53990b5c606708efa7b343e6df96f9a"} Sep 10 03:48:16.725998 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:16.725989 2562 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 10 03:48:16.740790 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:16.740738 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8hwtj" podStartSLOduration=4.403159142 podStartE2EDuration="22.740713105s" totalImagesPullingTime="18.337553963s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 03:47:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:48:16.74050903 +0000 UTC m=+23.616572182" watchObservedRunningTime="2026-09-10 03:48:16.740713105 +0000 UTC m=+23.616776256" Sep 10 03:48:17.442152 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:17.442113 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6ds5c" Sep 10 03:48:17.442729 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:17.442708 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6ds5c" Sep 10 03:48:17.598996 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:17.598960 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2p9b8" Sep 10 03:48:17.598996 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:17.599002 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gwzk7" Sep 10 03:48:17.599244 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:17.598966 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hlv2m" Sep 10 03:48:17.599244 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:17.599108 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2p9b8" podUID="158bac16-c713-47cd-91d7-9fc36d96f570" Sep 10 03:48:17.599244 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:17.599210 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hlv2m" podUID="10447409-1646-48fb-a5c9-494670b7d232" Sep 10 03:48:17.599413 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:17.599313 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gwzk7" podUID="348bacd3-9294-48e6-bbf4-60727b9a7bfb" Sep 10 03:48:17.732385 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:17.732283 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fl7hj_8459a996-8d64-4c23-9e92-1a5bcca253b5/ovn-acl-logging/0.log" Sep 10 03:48:17.733154 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:17.733126 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" event={"ID":"8459a996-8d64-4c23-9e92-1a5bcca253b5","Type":"ContainerStarted","Data":"aec9dec33621856fbd480d43542ba2f32cf91aaa0eb7b02da22dccea63ad9e4c"} Sep 10 03:48:17.735623 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:17.735594 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkxrh" event={"ID":"434f994a-63b7-4290-a72a-eb55456f4790","Type":"ContainerStarted","Data":"efde919c2daac31c53531fe56efcf0584cc0aee6146caee8a74f060448cf5790"} Sep 10 03:48:17.752310 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:17.752243 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jkxrh" podStartSLOduration=3.144382815 podStartE2EDuration="23.752228132s" totalImagesPullingTime="20.607845317s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 03:47:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:48:17.752012327 +0000 UTC m=+24.628075479" watchObservedRunningTime="2026-09-10 03:48:17.752228132 +0000 UTC m=+24.628291284" Sep 10 03:48:19.598787 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:19.598568 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gwzk7" Sep 10 03:48:19.599275 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:19.598568 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hlv2m" Sep 10 03:48:19.599275 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:19.598907 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gwzk7" podUID="348bacd3-9294-48e6-bbf4-60727b9a7bfb" Sep 10 03:48:19.599275 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:19.598986 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hlv2m" podUID="10447409-1646-48fb-a5c9-494670b7d232" Sep 10 03:48:19.599275 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:19.598568 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2p9b8" Sep 10 03:48:19.599275 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:19.599091 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2p9b8" podUID="158bac16-c713-47cd-91d7-9fc36d96f570" Sep 10 03:48:19.745914 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:19.745881 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fl7hj_8459a996-8d64-4c23-9e92-1a5bcca253b5/ovn-acl-logging/0.log" Sep 10 03:48:19.746423 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:19.746384 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" event={"ID":"8459a996-8d64-4c23-9e92-1a5bcca253b5","Type":"ContainerStarted","Data":"77dd4cf5c14c7eba4a4c6557a56cfc67be2bec179eb1a2382f4d24d8015a9a27"} Sep 10 03:48:19.746869 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:19.746791 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:48:19.747004 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:19.746975 2562 scope.go:117] "RemoveContainer" containerID="fe31bd49803112d5d592fad1d17dd5506bf8a9b63bf7dbe67bf7b0d06253d2ff" Sep 10 03:48:19.748583 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:19.748555 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ckqrf" event={"ID":"aae7f13d-be0c-4bd9-af61-40fc05f144c0","Type":"ContainerStarted","Data":"79eb4b1eaa392a7e22da8d78a11cc57f0c7bdca382410e9bd34b516e3ae64d99"} Sep 10 03:48:19.765440 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:19.765415 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:48:20.757821 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:20.757795 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fl7hj_8459a996-8d64-4c23-9e92-1a5bcca253b5/ovn-acl-logging/0.log" Sep 10 03:48:20.758484 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:20.758189 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" event={"ID":"8459a996-8d64-4c23-9e92-1a5bcca253b5","Type":"ContainerStarted","Data":"90cb58fe3c0aa8317da3f2e2e0a98429d686c458fc9fed1a3ce51ce7163afab3"} Sep 10 03:48:20.758484 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:20.758446 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:48:20.758602 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:20.758515 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:48:20.760142 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:20.760115 2562 generic.go:356] "Generic (PLEG): container finished" podID="aae7f13d-be0c-4bd9-af61-40fc05f144c0" containerID="79eb4b1eaa392a7e22da8d78a11cc57f0c7bdca382410e9bd34b516e3ae64d99" exitCode=0 Sep 10 03:48:20.760258 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:20.760156 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ckqrf" event={"ID":"aae7f13d-be0c-4bd9-af61-40fc05f144c0","Type":"ContainerDied","Data":"79eb4b1eaa392a7e22da8d78a11cc57f0c7bdca382410e9bd34b516e3ae64d99"} Sep 10 03:48:20.774465 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:20.774438 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:48:20.783152 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:20.783058 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" podStartSLOduration=8.366207193 podStartE2EDuration="26.783043331s" totalImagesPullingTime="18.416836138s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 03:47:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:48:20.782911432 +0000 UTC m=+27.658974584" watchObservedRunningTime="2026-09-10 03:48:20.783043331 +0000 UTC m=+27.659106483" Sep 10 03:48:21.507351 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:21.507143 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2p9b8"] Sep 10 03:48:21.507535 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:21.507473 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2p9b8" Sep 10 03:48:21.507617 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:21.507594 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2p9b8" podUID="158bac16-c713-47cd-91d7-9fc36d96f570" Sep 10 03:48:21.510147 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:21.510118 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hlv2m"] Sep 10 03:48:21.510325 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:21.510242 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hlv2m" Sep 10 03:48:21.510379 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:21.510346 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hlv2m" podUID="10447409-1646-48fb-a5c9-494670b7d232" Sep 10 03:48:21.510948 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:21.510922 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gwzk7"] Sep 10 03:48:21.511180 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:21.511027 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gwzk7" Sep 10 03:48:21.511180 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:21.511122 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gwzk7" podUID="348bacd3-9294-48e6-bbf4-60727b9a7bfb" Sep 10 03:48:22.766142 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:22.766109 2562 generic.go:356] "Generic (PLEG): container finished" podID="aae7f13d-be0c-4bd9-af61-40fc05f144c0" containerID="efb45f2aa793dd984a891863347d1e4110736cb4ac2f06161dee35baa93d16e7" exitCode=0 Sep 10 03:48:22.766548 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:22.766411 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ckqrf" event={"ID":"aae7f13d-be0c-4bd9-af61-40fc05f144c0","Type":"ContainerDied","Data":"efb45f2aa793dd984a891863347d1e4110736cb4ac2f06161dee35baa93d16e7"} Sep 10 03:48:23.599410 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:23.599374 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hlv2m" Sep 10 03:48:23.599523 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:23.599484 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2p9b8" Sep 10 03:48:23.599523 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:23.599511 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gwzk7" Sep 10 03:48:23.599608 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:23.599477 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hlv2m" podUID="10447409-1646-48fb-a5c9-494670b7d232" Sep 10 03:48:23.599642 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:23.599619 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2p9b8" podUID="158bac16-c713-47cd-91d7-9fc36d96f570" Sep 10 03:48:23.599705 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:23.599687 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gwzk7" podUID="348bacd3-9294-48e6-bbf4-60727b9a7bfb" Sep 10 03:48:23.771343 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:23.771306 2562 generic.go:356] "Generic (PLEG): container finished" podID="aae7f13d-be0c-4bd9-af61-40fc05f144c0" containerID="3dead0e1b860078b1bf5aae76d237abd623d4e68f8903817c1c1ce7b98f2e74e" exitCode=0 Sep 10 03:48:23.771343 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:23.771321 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ckqrf" event={"ID":"aae7f13d-be0c-4bd9-af61-40fc05f144c0","Type":"ContainerDied","Data":"3dead0e1b860078b1bf5aae76d237abd623d4e68f8903817c1c1ce7b98f2e74e"} Sep 10 03:48:25.598722 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:25.598679 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2p9b8" Sep 10 03:48:25.599162 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:25.598679 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gwzk7" Sep 10 03:48:25.599162 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:25.598839 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2p9b8" podUID="158bac16-c713-47cd-91d7-9fc36d96f570" Sep 10 03:48:25.599162 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:25.598685 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hlv2m" Sep 10 03:48:25.599162 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:25.598917 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gwzk7" podUID="348bacd3-9294-48e6-bbf4-60727b9a7bfb" Sep 10 03:48:25.599162 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:25.599006 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hlv2m" podUID="10447409-1646-48fb-a5c9-494670b7d232" Sep 10 03:48:27.338608 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.338413 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-74.ec2.internal" event="NodeReady" Sep 10 03:48:27.339013 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.338724 2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 10 03:48:27.342765 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.342742 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158bac16-c713-47cd-91d7-9fc36d96f570-metrics-certs\") pod \"network-metrics-daemon-2p9b8\" (UID: \"158bac16-c713-47cd-91d7-9fc36d96f570\") " pod="openshift-multus/network-metrics-daemon-2p9b8" Sep 10 03:48:27.342820 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.342792 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/348bacd3-9294-48e6-bbf4-60727b9a7bfb-original-pull-secret\") pod \"global-pull-secret-syncer-gwzk7\" (UID: \"348bacd3-9294-48e6-bbf4-60727b9a7bfb\") " pod="kube-system/global-pull-secret-syncer-gwzk7" Sep 10 03:48:27.342885 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:27.342874 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 03:48:27.342934 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:27.342926 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/348bacd3-9294-48e6-bbf4-60727b9a7bfb-original-pull-secret podName:348bacd3-9294-48e6-bbf4-60727b9a7bfb nodeName:}" failed. No retries permitted until 2026-09-10 03:48:59.34291344 +0000 UTC m=+66.218976569 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/348bacd3-9294-48e6-bbf4-60727b9a7bfb-original-pull-secret") pod "global-pull-secret-syncer-gwzk7" (UID: "348bacd3-9294-48e6-bbf4-60727b9a7bfb") : object "kube-system"/"original-pull-secret" not registered Sep 10 03:48:27.342976 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:27.342877 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 03:48:27.343020 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:27.343009 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/158bac16-c713-47cd-91d7-9fc36d96f570-metrics-certs podName:158bac16-c713-47cd-91d7-9fc36d96f570 nodeName:}" failed. No retries permitted until 2026-09-10 03:48:59.342992091 +0000 UTC m=+66.219055221 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/158bac16-c713-47cd-91d7-9fc36d96f570-metrics-certs") pod "network-metrics-daemon-2p9b8" (UID: "158bac16-c713-47cd-91d7-9fc36d96f570") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 03:48:27.375027 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.374994 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-554d6765f7-ss9hs"] Sep 10 03:48:27.443411 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.443373 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhrqr\" (UniqueName: \"kubernetes.io/projected/10447409-1646-48fb-a5c9-494670b7d232-kube-api-access-rhrqr\") pod \"network-check-target-hlv2m\" (UID: \"10447409-1646-48fb-a5c9-494670b7d232\") " pod="openshift-network-diagnostics/network-check-target-hlv2m" Sep 10 03:48:27.443594 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:27.443502 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 03:48:27.443594 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:27.443517 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 03:48:27.443594 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:27.443526 2562 projected.go:196] Error preparing data for projected volume kube-api-access-rhrqr for pod openshift-network-diagnostics/network-check-target-hlv2m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 03:48:27.443594 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:27.443573 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10447409-1646-48fb-a5c9-494670b7d232-kube-api-access-rhrqr podName:10447409-1646-48fb-a5c9-494670b7d232 nodeName:}" failed. No retries permitted until 2026-09-10 03:48:59.443559487 +0000 UTC m=+66.319622617 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rhrqr" (UniqueName: "kubernetes.io/projected/10447409-1646-48fb-a5c9-494670b7d232-kube-api-access-rhrqr") pod "network-check-target-hlv2m" (UID: "10447409-1646-48fb-a5c9-494670b7d232") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 03:48:27.449980 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.449952 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-2mjrb"] Sep 10 03:48:27.450092 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.450077 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:48:27.452305 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.452265 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 10 03:48:27.452432 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.452338 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 10 03:48:27.452432 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.452382 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 10 03:48:27.452910 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.452881 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-lfmcw\"" Sep 10 03:48:27.460331 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.459877 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 10 03:48:27.463948 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.463926 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-dgzqq"] Sep 10 03:48:27.464047 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.464031 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-2mjrb" Sep 10 03:48:27.466262 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.466241 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Sep 10 03:48:27.466391 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.466369 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-qn2w9\"" Sep 10 03:48:27.466451 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.466407 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Sep 10 03:48:27.482499 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.482457 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dkkkk"] Sep 10 03:48:27.482660 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.482535 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-dgzqq" Sep 10 03:48:27.485059 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.485034 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 10 03:48:27.485059 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.485053 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 10 03:48:27.485225 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.485054 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-p8h64\"" Sep 10 03:48:27.502711 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.502682 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-554d6765f7-ss9hs"] Sep 10 03:48:27.502711 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.502718 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-kx9jc"] Sep 10 03:48:27.502967 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.502818 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dkkkk" Sep 10 03:48:27.505126 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.505101 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 10 03:48:27.505253 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.505149 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 10 03:48:27.505253 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.505215 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bt2fn\"" Sep 10 03:48:27.518899 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.518785 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-dgzqq"] Sep 10 03:48:27.518899 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.518825 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-2mjrb"] Sep 10 03:48:27.518899 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.518847 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kx9jc"] Sep 10 03:48:27.518899 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.518865 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dkkkk"] Sep 10 03:48:27.519249 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.518979 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kx9jc" Sep 10 03:48:27.522589 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.522561 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 10 03:48:27.522770 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.522644 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 10 03:48:27.522770 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.522676 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-cprh9\"" Sep 10 03:48:27.522884 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.522796 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 10 03:48:27.528422 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.528391 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:48:27.544222 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.544178 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-certificates\") pod \"image-registry-554d6765f7-ss9hs\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:48:27.544428 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.544230 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-image-registry-private-configuration\") pod \"image-registry-554d6765f7-ss9hs\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:48:27.544428 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.544342 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/aa9e7159-181a-4a1f-afe1-33cd08e9ba6a-nginx-conf\") pod \"networking-console-plugin-79977f9844-2mjrb\" (UID: \"aa9e7159-181a-4a1f-afe1-33cd08e9ba6a\") " pod="openshift-network-console/networking-console-plugin-79977f9844-2mjrb" Sep 10 03:48:27.544428 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.544369 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-ca-trust-extracted\") pod \"image-registry-554d6765f7-ss9hs\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:48:27.544428 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.544392 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-installation-pull-secrets\") pod \"image-registry-554d6765f7-ss9hs\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:48:27.544637 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.544450 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-bound-sa-token\") pod \"image-registry-554d6765f7-ss9hs\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:48:27.544637 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.544488 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-tls\") pod \"image-registry-554d6765f7-ss9hs\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:48:27.544637 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.544515 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfg5p\" (UniqueName: \"kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-kube-api-access-sfg5p\") pod \"image-registry-554d6765f7-ss9hs\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:48:27.544637 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.544547 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-trusted-ca\") pod \"image-registry-554d6765f7-ss9hs\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:48:27.544797 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.544636 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/aa9e7159-181a-4a1f-afe1-33cd08e9ba6a-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-2mjrb\" (UID: \"aa9e7159-181a-4a1f-afe1-33cd08e9ba6a\") " pod="openshift-network-console/networking-console-plugin-79977f9844-2mjrb" Sep 10 03:48:27.544797 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.544664 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxgc7\" (UniqueName: \"kubernetes.io/projected/a7b43134-8b96-4e81-adeb-6fcf187f9bbf-kube-api-access-gxgc7\") pod \"network-check-source-685456655f-dgzqq\" (UID: \"a7b43134-8b96-4e81-adeb-6fcf187f9bbf\") " pod="openshift-network-diagnostics/network-check-source-685456655f-dgzqq" Sep 10 03:48:27.607159 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.607127 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gwzk7" Sep 10 03:48:27.607366 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.607127 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hlv2m" Sep 10 03:48:27.607366 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.607127 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2p9b8" Sep 10 03:48:27.609693 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.609467 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 03:48:27.609693 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.609481 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2jbt8\"" Sep 10 03:48:27.609693 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.609491 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 10 03:48:27.609693 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.609481 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7r79s\"" Sep 10 03:48:27.630106 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.630081 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:48:27.645601 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.645574 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76d9w\" (UniqueName: \"kubernetes.io/projected/e6305575-8975-44bd-bd29-866c8457a56c-kube-api-access-76d9w\") pod \"ingress-canary-kx9jc\" (UID: \"e6305575-8975-44bd-bd29-866c8457a56c\") " pod="openshift-ingress-canary/ingress-canary-kx9jc" Sep 10 03:48:27.645763 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.645612 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6305575-8975-44bd-bd29-866c8457a56c-cert\") pod \"ingress-canary-kx9jc\" (UID: \"e6305575-8975-44bd-bd29-866c8457a56c\") " pod="openshift-ingress-canary/ingress-canary-kx9jc" Sep 10 03:48:27.645763 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.645635 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/897fe719-a91a-4d75-8efb-4cb2fbade932-metrics-tls\") pod \"dns-default-dkkkk\" (UID: \"897fe719-a91a-4d75-8efb-4cb2fbade932\") " pod="openshift-dns/dns-default-dkkkk" Sep 10 03:48:27.645763 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.645656 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/897fe719-a91a-4d75-8efb-4cb2fbade932-tmp-dir\") pod \"dns-default-dkkkk\" (UID: \"897fe719-a91a-4d75-8efb-4cb2fbade932\") " pod="openshift-dns/dns-default-dkkkk" Sep 10 03:48:27.645763 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.645703 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-certificates\") pod \"image-registry-554d6765f7-ss9hs\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:48:27.645763 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.645729 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/897fe719-a91a-4d75-8efb-4cb2fbade932-config-volume\") pod \"dns-default-dkkkk\" (UID: \"897fe719-a91a-4d75-8efb-4cb2fbade932\") " pod="openshift-dns/dns-default-dkkkk" Sep 10 03:48:27.645763 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.645752 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-image-registry-private-configuration\") pod \"image-registry-554d6765f7-ss9hs\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:48:27.646048 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.645778 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/aa9e7159-181a-4a1f-afe1-33cd08e9ba6a-nginx-conf\") pod \"networking-console-plugin-79977f9844-2mjrb\" (UID: \"aa9e7159-181a-4a1f-afe1-33cd08e9ba6a\") " pod="openshift-network-console/networking-console-plugin-79977f9844-2mjrb" Sep 10 03:48:27.646048 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.645797 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-ca-trust-extracted\") pod \"image-registry-554d6765f7-ss9hs\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:48:27.646048 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.645825 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-installation-pull-secrets\") pod \"image-registry-554d6765f7-ss9hs\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:48:27.646048 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.645853 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-bound-sa-token\") pod \"image-registry-554d6765f7-ss9hs\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:48:27.646048 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.645882 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-tls\") pod \"image-registry-554d6765f7-ss9hs\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:48:27.646048 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.645908 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfg5p\" (UniqueName: \"kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-kube-api-access-sfg5p\") pod \"image-registry-554d6765f7-ss9hs\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:48:27.646048 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.645943 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-trusted-ca\") pod \"image-registry-554d6765f7-ss9hs\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:48:27.646048 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.645982 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87x66\" (UniqueName: \"kubernetes.io/projected/897fe719-a91a-4d75-8efb-4cb2fbade932-kube-api-access-87x66\") pod \"dns-default-dkkkk\" (UID: \"897fe719-a91a-4d75-8efb-4cb2fbade932\") " pod="openshift-dns/dns-default-dkkkk" Sep 10 03:48:27.646048 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.646015 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/aa9e7159-181a-4a1f-afe1-33cd08e9ba6a-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-2mjrb\" (UID: \"aa9e7159-181a-4a1f-afe1-33cd08e9ba6a\") " pod="openshift-network-console/networking-console-plugin-79977f9844-2mjrb" Sep 10 03:48:27.646048 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:27.646019 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 03:48:27.646048 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:27.646037 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-554d6765f7-ss9hs: secret "image-registry-tls" not found Sep 10 03:48:27.646048 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.646039 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxgc7\" (UniqueName: \"kubernetes.io/projected/a7b43134-8b96-4e81-adeb-6fcf187f9bbf-kube-api-access-gxgc7\") pod \"network-check-source-685456655f-dgzqq\" (UID: \"a7b43134-8b96-4e81-adeb-6fcf187f9bbf\") " pod="openshift-network-diagnostics/network-check-source-685456655f-dgzqq" Sep 10 03:48:27.646647 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:27.646091 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-tls podName:4ba38a7e-088d-4fdf-840c-c85b7826f1a6 nodeName:}" failed. No retries permitted until 2026-09-10 03:48:28.146073915 +0000 UTC m=+35.022137050 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-tls") pod "image-registry-554d6765f7-ss9hs" (UID: "4ba38a7e-088d-4fdf-840c-c85b7826f1a6") : secret "image-registry-tls" not found Sep 10 03:48:27.646647 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.646244 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-ca-trust-extracted\") pod \"image-registry-554d6765f7-ss9hs\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:48:27.646647 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.646401 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-certificates\") pod \"image-registry-554d6765f7-ss9hs\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:48:27.646647 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:27.646450 2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 03:48:27.646647 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:27.646511 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa9e7159-181a-4a1f-afe1-33cd08e9ba6a-networking-console-plugin-cert podName:aa9e7159-181a-4a1f-afe1-33cd08e9ba6a nodeName:}" failed. No retries permitted until 2026-09-10 03:48:28.14649473 +0000 UTC m=+35.022557894 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/aa9e7159-181a-4a1f-afe1-33cd08e9ba6a-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-2mjrb" (UID: "aa9e7159-181a-4a1f-afe1-33cd08e9ba6a") : secret "networking-console-plugin-cert" not found Sep 10 03:48:27.646647 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.646591 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/aa9e7159-181a-4a1f-afe1-33cd08e9ba6a-nginx-conf\") pod \"networking-console-plugin-79977f9844-2mjrb\" (UID: \"aa9e7159-181a-4a1f-afe1-33cd08e9ba6a\") " pod="openshift-network-console/networking-console-plugin-79977f9844-2mjrb" Sep 10 03:48:27.647082 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.647055 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-trusted-ca\") pod \"image-registry-554d6765f7-ss9hs\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:48:27.650230 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.650210 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-installation-pull-secrets\") pod \"image-registry-554d6765f7-ss9hs\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:48:27.650353 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.650241 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-image-registry-private-configuration\") pod \"image-registry-554d6765f7-ss9hs\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:48:27.656575 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.656553 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-bound-sa-token\") pod \"image-registry-554d6765f7-ss9hs\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:48:27.656944 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.656920 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfg5p\" (UniqueName: \"kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-kube-api-access-sfg5p\") pod \"image-registry-554d6765f7-ss9hs\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:48:27.657058 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.656923 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxgc7\" (UniqueName: \"kubernetes.io/projected/a7b43134-8b96-4e81-adeb-6fcf187f9bbf-kube-api-access-gxgc7\") pod \"network-check-source-685456655f-dgzqq\" (UID: \"a7b43134-8b96-4e81-adeb-6fcf187f9bbf\") " pod="openshift-network-diagnostics/network-check-source-685456655f-dgzqq" Sep 10 03:48:27.746442 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.746405 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87x66\" (UniqueName: \"kubernetes.io/projected/897fe719-a91a-4d75-8efb-4cb2fbade932-kube-api-access-87x66\") pod \"dns-default-dkkkk\" (UID: \"897fe719-a91a-4d75-8efb-4cb2fbade932\") " pod="openshift-dns/dns-default-dkkkk" Sep 10 03:48:27.746641 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.746478 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76d9w\" (UniqueName: \"kubernetes.io/projected/e6305575-8975-44bd-bd29-866c8457a56c-kube-api-access-76d9w\") pod \"ingress-canary-kx9jc\" (UID: \"e6305575-8975-44bd-bd29-866c8457a56c\") " pod="openshift-ingress-canary/ingress-canary-kx9jc" Sep 10 03:48:27.746641 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.746502 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6305575-8975-44bd-bd29-866c8457a56c-cert\") pod \"ingress-canary-kx9jc\" (UID: \"e6305575-8975-44bd-bd29-866c8457a56c\") " pod="openshift-ingress-canary/ingress-canary-kx9jc" Sep 10 03:48:27.746641 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.746522 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/897fe719-a91a-4d75-8efb-4cb2fbade932-metrics-tls\") pod \"dns-default-dkkkk\" (UID: \"897fe719-a91a-4d75-8efb-4cb2fbade932\") " pod="openshift-dns/dns-default-dkkkk" Sep 10 03:48:27.746641 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.746584 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/897fe719-a91a-4d75-8efb-4cb2fbade932-tmp-dir\") pod \"dns-default-dkkkk\" (UID: \"897fe719-a91a-4d75-8efb-4cb2fbade932\") " pod="openshift-dns/dns-default-dkkkk" Sep 10 03:48:27.746641 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.746628 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/897fe719-a91a-4d75-8efb-4cb2fbade932-config-volume\") pod \"dns-default-dkkkk\" (UID: \"897fe719-a91a-4d75-8efb-4cb2fbade932\") " pod="openshift-dns/dns-default-dkkkk" Sep 10 03:48:27.746893 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:27.746847 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 03:48:27.746949 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:27.746922 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/897fe719-a91a-4d75-8efb-4cb2fbade932-metrics-tls podName:897fe719-a91a-4d75-8efb-4cb2fbade932 nodeName:}" failed. No retries permitted until 2026-09-10 03:48:28.24690072 +0000 UTC m=+35.122963854 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/897fe719-a91a-4d75-8efb-4cb2fbade932-metrics-tls") pod "dns-default-dkkkk" (UID: "897fe719-a91a-4d75-8efb-4cb2fbade932") : secret "dns-default-metrics-tls" not found Sep 10 03:48:27.747087 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.747065 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/897fe719-a91a-4d75-8efb-4cb2fbade932-tmp-dir\") pod \"dns-default-dkkkk\" (UID: \"897fe719-a91a-4d75-8efb-4cb2fbade932\") " pod="openshift-dns/dns-default-dkkkk" Sep 10 03:48:27.747307 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.747268 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/897fe719-a91a-4d75-8efb-4cb2fbade932-config-volume\") pod \"dns-default-dkkkk\" (UID: \"897fe719-a91a-4d75-8efb-4cb2fbade932\") " pod="openshift-dns/dns-default-dkkkk" Sep 10 03:48:27.749082 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.749065 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6305575-8975-44bd-bd29-866c8457a56c-cert\") pod \"ingress-canary-kx9jc\" (UID: \"e6305575-8975-44bd-bd29-866c8457a56c\") " pod="openshift-ingress-canary/ingress-canary-kx9jc" Sep 10 03:48:27.754888 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.754852 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87x66\" (UniqueName: \"kubernetes.io/projected/897fe719-a91a-4d75-8efb-4cb2fbade932-kube-api-access-87x66\") pod \"dns-default-dkkkk\" (UID: \"897fe719-a91a-4d75-8efb-4cb2fbade932\") " pod="openshift-dns/dns-default-dkkkk" Sep 10 03:48:27.755419 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.755397 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76d9w\" (UniqueName: \"kubernetes.io/projected/e6305575-8975-44bd-bd29-866c8457a56c-kube-api-access-76d9w\") pod \"ingress-canary-kx9jc\" (UID: \"e6305575-8975-44bd-bd29-866c8457a56c\") " pod="openshift-ingress-canary/ingress-canary-kx9jc" Sep 10 03:48:27.791002 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.790918 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-dgzqq" Sep 10 03:48:27.792223 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.792185 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:48:27.828118 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.828079 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kx9jc" Sep 10 03:48:27.892829 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.892790 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:48:27.993982 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:27.993942 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:48:28.122789 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:28.122697 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:48:28.149639 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:28.149606 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-tls\") pod \"image-registry-554d6765f7-ss9hs\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:48:28.149838 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:28.149656 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/aa9e7159-181a-4a1f-afe1-33cd08e9ba6a-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-2mjrb\" (UID: \"aa9e7159-181a-4a1f-afe1-33cd08e9ba6a\") " pod="openshift-network-console/networking-console-plugin-79977f9844-2mjrb" Sep 10 03:48:28.149838 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:28.149776 2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 03:48:28.149838 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:28.149782 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 03:48:28.149838 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:28.149803 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-554d6765f7-ss9hs: secret "image-registry-tls" not found Sep 10 03:48:28.150037 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:28.149849 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa9e7159-181a-4a1f-afe1-33cd08e9ba6a-networking-console-plugin-cert podName:aa9e7159-181a-4a1f-afe1-33cd08e9ba6a nodeName:}" failed. No retries permitted until 2026-09-10 03:48:29.149828008 +0000 UTC m=+36.025891159 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/aa9e7159-181a-4a1f-afe1-33cd08e9ba6a-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-2mjrb" (UID: "aa9e7159-181a-4a1f-afe1-33cd08e9ba6a") : secret "networking-console-plugin-cert" not found Sep 10 03:48:28.150037 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:28.149865 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-tls podName:4ba38a7e-088d-4fdf-840c-c85b7826f1a6 nodeName:}" failed. No retries permitted until 2026-09-10 03:48:29.149858612 +0000 UTC m=+36.025921865 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-tls") pod "image-registry-554d6765f7-ss9hs" (UID: "4ba38a7e-088d-4fdf-840c-c85b7826f1a6") : secret "image-registry-tls" not found Sep 10 03:48:28.222857 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:28.222823 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:48:28.250160 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:28.250122 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/897fe719-a91a-4d75-8efb-4cb2fbade932-metrics-tls\") pod \"dns-default-dkkkk\" (UID: \"897fe719-a91a-4d75-8efb-4cb2fbade932\") " pod="openshift-dns/dns-default-dkkkk" Sep 10 03:48:28.250356 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:28.250306 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 03:48:28.250413 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:28.250384 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/897fe719-a91a-4d75-8efb-4cb2fbade932-metrics-tls podName:897fe719-a91a-4d75-8efb-4cb2fbade932 nodeName:}" failed. No retries permitted until 2026-09-10 03:48:29.250363781 +0000 UTC m=+36.126426914 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/897fe719-a91a-4d75-8efb-4cb2fbade932-metrics-tls") pod "dns-default-dkkkk" (UID: "897fe719-a91a-4d75-8efb-4cb2fbade932") : secret "dns-default-metrics-tls" not found Sep 10 03:48:29.158629 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:29.158584 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-tls\") pod \"image-registry-554d6765f7-ss9hs\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:48:29.159463 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:29.158654 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/aa9e7159-181a-4a1f-afe1-33cd08e9ba6a-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-2mjrb\" (UID: \"aa9e7159-181a-4a1f-afe1-33cd08e9ba6a\") " pod="openshift-network-console/networking-console-plugin-79977f9844-2mjrb" Sep 10 03:48:29.159463 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:29.158746 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 03:48:29.159463 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:29.158762 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-554d6765f7-ss9hs: secret "image-registry-tls" not found Sep 10 03:48:29.159463 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:29.158842 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-tls podName:4ba38a7e-088d-4fdf-840c-c85b7826f1a6 nodeName:}" failed. No retries permitted until 2026-09-10 03:48:31.158826926 +0000 UTC m=+38.034890070 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-tls") pod "image-registry-554d6765f7-ss9hs" (UID: "4ba38a7e-088d-4fdf-840c-c85b7826f1a6") : secret "image-registry-tls" not found Sep 10 03:48:29.159463 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:29.158877 2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 03:48:29.159463 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:29.158954 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa9e7159-181a-4a1f-afe1-33cd08e9ba6a-networking-console-plugin-cert podName:aa9e7159-181a-4a1f-afe1-33cd08e9ba6a nodeName:}" failed. No retries permitted until 2026-09-10 03:48:31.158933639 +0000 UTC m=+38.034996782 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/aa9e7159-181a-4a1f-afe1-33cd08e9ba6a-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-2mjrb" (UID: "aa9e7159-181a-4a1f-afe1-33cd08e9ba6a") : secret "networking-console-plugin-cert" not found Sep 10 03:48:29.183032 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:29.182996 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:48:29.245829 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:29.245799 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:48:29.259907 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:29.259874 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/897fe719-a91a-4d75-8efb-4cb2fbade932-metrics-tls\") pod \"dns-default-dkkkk\" (UID: \"897fe719-a91a-4d75-8efb-4cb2fbade932\") " pod="openshift-dns/dns-default-dkkkk" Sep 10 03:48:29.260020 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:29.259994 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 03:48:29.260063 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:29.260048 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/897fe719-a91a-4d75-8efb-4cb2fbade932-metrics-tls podName:897fe719-a91a-4d75-8efb-4cb2fbade932 nodeName:}" failed. No retries permitted until 2026-09-10 03:48:31.260033212 +0000 UTC m=+38.136096341 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/897fe719-a91a-4d75-8efb-4cb2fbade932-metrics-tls") pod "dns-default-dkkkk" (UID: "897fe719-a91a-4d75-8efb-4cb2fbade932") : secret "dns-default-metrics-tls" not found Sep 10 03:48:29.333524 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:29.333427 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kx9jc"] Sep 10 03:48:29.335692 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:29.335670 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-dgzqq"] Sep 10 03:48:29.339078 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:48:29.339049 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6305575_8975_44bd_bd29_866c8457a56c.slice/crio-9cb6bdbee3e6144608c4b83701fc987a46c368cdfd6dfc2d846db218760029ff WatchSource:0}: Error finding container 9cb6bdbee3e6144608c4b83701fc987a46c368cdfd6dfc2d846db218760029ff: Status 404 returned error can't find the container with id 9cb6bdbee3e6144608c4b83701fc987a46c368cdfd6dfc2d846db218760029ff Sep 10 03:48:29.339811 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:48:29.339786 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7b43134_8b96_4e81_adeb_6fcf187f9bbf.slice/crio-ca9ec5772653bee729e2ea3e8d8dfdc13a7ccd254459f97f7bc9527099fa08f4 WatchSource:0}: Error finding container ca9ec5772653bee729e2ea3e8d8dfdc13a7ccd254459f97f7bc9527099fa08f4: Status 404 returned error can't find the container with id ca9ec5772653bee729e2ea3e8d8dfdc13a7ccd254459f97f7bc9527099fa08f4 Sep 10 03:48:29.728773 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:29.728567 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:48:29.781417 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:29.781385 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:48:29.785923 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:29.785888 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-dgzqq" event={"ID":"a7b43134-8b96-4e81-adeb-6fcf187f9bbf","Type":"ContainerStarted","Data":"ca9ec5772653bee729e2ea3e8d8dfdc13a7ccd254459f97f7bc9527099fa08f4"} Sep 10 03:48:29.788603 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:29.788576 2562 generic.go:356] "Generic (PLEG): container finished" podID="aae7f13d-be0c-4bd9-af61-40fc05f144c0" containerID="03bd478723b80d4e01b987f4b91ddad3bcaf0f2d287c303be738e2536efdb346" exitCode=0 Sep 10 03:48:29.788782 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:29.788646 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ckqrf" event={"ID":"aae7f13d-be0c-4bd9-af61-40fc05f144c0","Type":"ContainerDied","Data":"03bd478723b80d4e01b987f4b91ddad3bcaf0f2d287c303be738e2536efdb346"} Sep 10 03:48:29.789942 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:29.789917 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kx9jc" event={"ID":"e6305575-8975-44bd-bd29-866c8457a56c","Type":"ContainerStarted","Data":"9cb6bdbee3e6144608c4b83701fc987a46c368cdfd6dfc2d846db218760029ff"} Sep 10 03:48:30.185211 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:30.185146 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:48:30.522662 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:30.522580 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:48:30.627168 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:30.627126 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:48:30.796958 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:30.796874 2562 generic.go:356] "Generic (PLEG): container finished" podID="aae7f13d-be0c-4bd9-af61-40fc05f144c0" containerID="d61fa58d447fa6b2bf45ccbbd2e3f9f2ab9b80aec55be5d91a2b07a0f461b9f2" exitCode=0 Sep 10 03:48:30.796958 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:30.796910 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ckqrf" event={"ID":"aae7f13d-be0c-4bd9-af61-40fc05f144c0","Type":"ContainerDied","Data":"d61fa58d447fa6b2bf45ccbbd2e3f9f2ab9b80aec55be5d91a2b07a0f461b9f2"} Sep 10 03:48:31.176273 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:31.176226 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/aa9e7159-181a-4a1f-afe1-33cd08e9ba6a-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-2mjrb\" (UID: \"aa9e7159-181a-4a1f-afe1-33cd08e9ba6a\") " pod="openshift-network-console/networking-console-plugin-79977f9844-2mjrb" Sep 10 03:48:31.176446 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:31.176387 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-tls\") pod \"image-registry-554d6765f7-ss9hs\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:48:31.176495 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:31.176463 2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 03:48:31.176555 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:31.176535 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 03:48:31.176601 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:31.176560 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-554d6765f7-ss9hs: secret "image-registry-tls" not found Sep 10 03:48:31.176601 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:31.176549 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa9e7159-181a-4a1f-afe1-33cd08e9ba6a-networking-console-plugin-cert podName:aa9e7159-181a-4a1f-afe1-33cd08e9ba6a nodeName:}" failed. No retries permitted until 2026-09-10 03:48:35.176527135 +0000 UTC m=+42.052590267 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/aa9e7159-181a-4a1f-afe1-33cd08e9ba6a-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-2mjrb" (UID: "aa9e7159-181a-4a1f-afe1-33cd08e9ba6a") : secret "networking-console-plugin-cert" not found Sep 10 03:48:31.176678 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:31.176619 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-tls podName:4ba38a7e-088d-4fdf-840c-c85b7826f1a6 nodeName:}" failed. No retries permitted until 2026-09-10 03:48:35.176603254 +0000 UTC m=+42.052666398 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-tls") pod "image-registry-554d6765f7-ss9hs" (UID: "4ba38a7e-088d-4fdf-840c-c85b7826f1a6") : secret "image-registry-tls" not found Sep 10 03:48:31.277393 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:31.277353 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/897fe719-a91a-4d75-8efb-4cb2fbade932-metrics-tls\") pod \"dns-default-dkkkk\" (UID: \"897fe719-a91a-4d75-8efb-4cb2fbade932\") " pod="openshift-dns/dns-default-dkkkk" Sep 10 03:48:31.277843 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:31.277559 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 03:48:31.277843 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:31.277631 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/897fe719-a91a-4d75-8efb-4cb2fbade932-metrics-tls podName:897fe719-a91a-4d75-8efb-4cb2fbade932 nodeName:}" failed. No retries permitted until 2026-09-10 03:48:35.277610451 +0000 UTC m=+42.153673598 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/897fe719-a91a-4d75-8efb-4cb2fbade932-metrics-tls") pod "dns-default-dkkkk" (UID: "897fe719-a91a-4d75-8efb-4cb2fbade932") : secret "dns-default-metrics-tls" not found Sep 10 03:48:31.804387 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:31.804343 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ckqrf" event={"ID":"aae7f13d-be0c-4bd9-af61-40fc05f144c0","Type":"ContainerStarted","Data":"ed78e6dd328baea2a9aee2816d72c0e11f71cadf6d32b844f7a37439d58aa375"} Sep 10 03:48:31.826998 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:31.826942 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ckqrf" podStartSLOduration=7.143222094 podStartE2EDuration="37.826927064s" totalImagesPullingTime="30.68370497s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 03:47:54 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:48:31.826239916 +0000 UTC m=+38.702303080" watchObservedRunningTime="2026-09-10 03:48:31.826927064 +0000 UTC m=+38.702990215" Sep 10 03:48:31.951465 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:31.951433 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:48:32.004398 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:32.004356 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:48:33.005780 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:33.005745 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:48:33.106755 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:33.106723 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:48:33.811108 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:33.811069 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-dgzqq" event={"ID":"a7b43134-8b96-4e81-adeb-6fcf187f9bbf","Type":"ContainerStarted","Data":"640bc31720758bca1c24f985938cd9c8f1782bea589a42956f888f643b383bb2"} Sep 10 03:48:33.812686 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:33.812654 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kx9jc" event={"ID":"e6305575-8975-44bd-bd29-866c8457a56c","Type":"ContainerStarted","Data":"8f166d5283789168a1b143632bda8ecbbc230bb7e8d48303d7ef702f82d102eb"} Sep 10 03:48:33.826349 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:33.826286 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-dgzqq" podStartSLOduration=35.137030481 podStartE2EDuration="38.826248486s" totalImagesPullingTime="3.689218005s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:47:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:48:33.826020333 +0000 UTC m=+40.702083487" watchObservedRunningTime="2026-09-10 03:48:33.826248486 +0000 UTC m=+40.702311639" Sep 10 03:48:33.839102 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:33.839048 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-kx9jc" podStartSLOduration=3.157472278 podStartE2EDuration="6.83902637s" totalImagesPullingTime="3.681554092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:48:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:48:33.838933165 +0000 UTC m=+40.714996318" watchObservedRunningTime="2026-09-10 03:48:33.83902637 +0000 UTC m=+40.715089523" Sep 10 03:48:34.803386 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:34.803340 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:48:35.211976 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:35.211935 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-tls\") pod \"image-registry-554d6765f7-ss9hs\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:48:35.212127 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:35.211992 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/aa9e7159-181a-4a1f-afe1-33cd08e9ba6a-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-2mjrb\" (UID: \"aa9e7159-181a-4a1f-afe1-33cd08e9ba6a\") " pod="openshift-network-console/networking-console-plugin-79977f9844-2mjrb" Sep 10 03:48:35.212127 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:35.212091 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 03:48:35.212127 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:35.212117 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-554d6765f7-ss9hs: secret "image-registry-tls" not found Sep 10 03:48:35.212244 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:35.212165 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-tls podName:4ba38a7e-088d-4fdf-840c-c85b7826f1a6 nodeName:}" failed. No retries permitted until 2026-09-10 03:48:43.212151198 +0000 UTC m=+50.088214340 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-tls") pod "image-registry-554d6765f7-ss9hs" (UID: "4ba38a7e-088d-4fdf-840c-c85b7826f1a6") : secret "image-registry-tls" not found Sep 10 03:48:35.212244 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:35.212104 2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 03:48:35.212244 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:35.212222 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa9e7159-181a-4a1f-afe1-33cd08e9ba6a-networking-console-plugin-cert podName:aa9e7159-181a-4a1f-afe1-33cd08e9ba6a nodeName:}" failed. No retries permitted until 2026-09-10 03:48:43.212210817 +0000 UTC m=+50.088273948 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/aa9e7159-181a-4a1f-afe1-33cd08e9ba6a-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-2mjrb" (UID: "aa9e7159-181a-4a1f-afe1-33cd08e9ba6a") : secret "networking-console-plugin-cert" not found Sep 10 03:48:35.312826 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:35.312791 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/897fe719-a91a-4d75-8efb-4cb2fbade932-metrics-tls\") pod \"dns-default-dkkkk\" (UID: \"897fe719-a91a-4d75-8efb-4cb2fbade932\") " pod="openshift-dns/dns-default-dkkkk" Sep 10 03:48:35.313016 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:35.312910 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 03:48:35.313016 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:35.312983 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/897fe719-a91a-4d75-8efb-4cb2fbade932-metrics-tls podName:897fe719-a91a-4d75-8efb-4cb2fbade932 nodeName:}" failed. No retries permitted until 2026-09-10 03:48:43.312967971 +0000 UTC m=+50.189031117 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/897fe719-a91a-4d75-8efb-4cb2fbade932-metrics-tls") pod "dns-default-dkkkk" (UID: "897fe719-a91a-4d75-8efb-4cb2fbade932") : secret "dns-default-metrics-tls" not found Sep 10 03:48:40.461523 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:40.461485 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:48:40.561599 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:40.561560 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:48:40.971143 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:40.971088 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:48:41.071348 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:41.071318 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:48:41.228842 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:41.228754 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:48:41.283211 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:41.283176 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:48:43.277536 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:43.277492 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-tls\") pod \"image-registry-554d6765f7-ss9hs\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:48:43.278012 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:43.277555 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/aa9e7159-181a-4a1f-afe1-33cd08e9ba6a-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-2mjrb\" (UID: \"aa9e7159-181a-4a1f-afe1-33cd08e9ba6a\") " pod="openshift-network-console/networking-console-plugin-79977f9844-2mjrb" Sep 10 03:48:43.278012 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:43.277667 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 03:48:43.278012 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:43.277683 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-554d6765f7-ss9hs: secret "image-registry-tls" not found Sep 10 03:48:43.278012 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:43.277709 2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 03:48:43.278012 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:43.277745 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-tls podName:4ba38a7e-088d-4fdf-840c-c85b7826f1a6 nodeName:}" failed. No retries permitted until 2026-09-10 03:48:59.277727802 +0000 UTC m=+66.153790947 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-tls") pod "image-registry-554d6765f7-ss9hs" (UID: "4ba38a7e-088d-4fdf-840c-c85b7826f1a6") : secret "image-registry-tls" not found Sep 10 03:48:43.278012 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:43.277784 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa9e7159-181a-4a1f-afe1-33cd08e9ba6a-networking-console-plugin-cert podName:aa9e7159-181a-4a1f-afe1-33cd08e9ba6a nodeName:}" failed. No retries permitted until 2026-09-10 03:48:59.277771026 +0000 UTC m=+66.153834155 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/aa9e7159-181a-4a1f-afe1-33cd08e9ba6a-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-2mjrb" (UID: "aa9e7159-181a-4a1f-afe1-33cd08e9ba6a") : secret "networking-console-plugin-cert" not found Sep 10 03:48:43.378208 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:43.378163 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/897fe719-a91a-4d75-8efb-4cb2fbade932-metrics-tls\") pod \"dns-default-dkkkk\" (UID: \"897fe719-a91a-4d75-8efb-4cb2fbade932\") " pod="openshift-dns/dns-default-dkkkk" Sep 10 03:48:43.378377 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:43.378303 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 03:48:43.378377 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:43.378365 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/897fe719-a91a-4d75-8efb-4cb2fbade932-metrics-tls podName:897fe719-a91a-4d75-8efb-4cb2fbade932 nodeName:}" failed. No retries permitted until 2026-09-10 03:48:59.378348092 +0000 UTC m=+66.254411228 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/897fe719-a91a-4d75-8efb-4cb2fbade932-metrics-tls") pod "dns-default-dkkkk" (UID: "897fe719-a91a-4d75-8efb-4cb2fbade932") : secret "dns-default-metrics-tls" not found Sep 10 03:48:44.258422 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:44.258383 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:48:44.356461 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:44.356418 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:48:44.507611 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:44.507558 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:48:44.608879 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:44.608788 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:48:45.817428 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:45.817394 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:48:45.869756 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:45.869708 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:48:46.490685 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:46.490648 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:48:46.545775 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:46.545734 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:48:52.777447 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:52.777412 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fl7hj" Sep 10 03:48:52.926286 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:52.926247 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:48:53.027236 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:53.027199 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:48:59.301256 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:59.301207 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-tls\") pod \"image-registry-554d6765f7-ss9hs\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:48:59.301672 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:59.301272 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/aa9e7159-181a-4a1f-afe1-33cd08e9ba6a-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-2mjrb\" (UID: \"aa9e7159-181a-4a1f-afe1-33cd08e9ba6a\") " pod="openshift-network-console/networking-console-plugin-79977f9844-2mjrb" Sep 10 03:48:59.301672 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:59.301410 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 03:48:59.301672 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:59.301437 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-554d6765f7-ss9hs: secret "image-registry-tls" not found Sep 10 03:48:59.301672 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:59.301490 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-tls podName:4ba38a7e-088d-4fdf-840c-c85b7826f1a6 nodeName:}" failed. No retries permitted until 2026-09-10 03:49:31.301474105 +0000 UTC m=+98.177537234 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-tls") pod "image-registry-554d6765f7-ss9hs" (UID: "4ba38a7e-088d-4fdf-840c-c85b7826f1a6") : secret "image-registry-tls" not found Sep 10 03:48:59.301672 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:59.301414 2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 03:48:59.301672 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:59.301547 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa9e7159-181a-4a1f-afe1-33cd08e9ba6a-networking-console-plugin-cert podName:aa9e7159-181a-4a1f-afe1-33cd08e9ba6a nodeName:}" failed. No retries permitted until 2026-09-10 03:49:31.301535483 +0000 UTC m=+98.177598613 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/aa9e7159-181a-4a1f-afe1-33cd08e9ba6a-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-2mjrb" (UID: "aa9e7159-181a-4a1f-afe1-33cd08e9ba6a") : secret "networking-console-plugin-cert" not found Sep 10 03:48:59.402148 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:59.402094 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158bac16-c713-47cd-91d7-9fc36d96f570-metrics-certs\") pod \"network-metrics-daemon-2p9b8\" (UID: \"158bac16-c713-47cd-91d7-9fc36d96f570\") " pod="openshift-multus/network-metrics-daemon-2p9b8" Sep 10 03:48:59.402148 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:59.402150 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/897fe719-a91a-4d75-8efb-4cb2fbade932-metrics-tls\") pod \"dns-default-dkkkk\" (UID: \"897fe719-a91a-4d75-8efb-4cb2fbade932\") " pod="openshift-dns/dns-default-dkkkk" Sep 10 03:48:59.402433 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:59.402212 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/348bacd3-9294-48e6-bbf4-60727b9a7bfb-original-pull-secret\") pod \"global-pull-secret-syncer-gwzk7\" (UID: \"348bacd3-9294-48e6-bbf4-60727b9a7bfb\") " pod="kube-system/global-pull-secret-syncer-gwzk7" Sep 10 03:48:59.402433 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:59.402352 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 03:48:59.402433 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:59.402402 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/897fe719-a91a-4d75-8efb-4cb2fbade932-metrics-tls podName:897fe719-a91a-4d75-8efb-4cb2fbade932 nodeName:}" failed. No retries permitted until 2026-09-10 03:49:31.402388824 +0000 UTC m=+98.278451957 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/897fe719-a91a-4d75-8efb-4cb2fbade932-metrics-tls") pod "dns-default-dkkkk" (UID: "897fe719-a91a-4d75-8efb-4cb2fbade932") : secret "dns-default-metrics-tls" not found Sep 10 03:48:59.404463 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:59.404443 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 03:48:59.404580 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:59.404563 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 10 03:48:59.412620 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:59.412596 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 10 03:48:59.412755 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:48:59.412678 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/158bac16-c713-47cd-91d7-9fc36d96f570-metrics-certs podName:158bac16-c713-47cd-91d7-9fc36d96f570 nodeName:}" failed. No retries permitted until 2026-09-10 03:50:03.412656707 +0000 UTC m=+130.288719841 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/158bac16-c713-47cd-91d7-9fc36d96f570-metrics-certs") pod "network-metrics-daemon-2p9b8" (UID: "158bac16-c713-47cd-91d7-9fc36d96f570") : secret "metrics-daemon-secret" not found Sep 10 03:48:59.415821 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:59.415798 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/348bacd3-9294-48e6-bbf4-60727b9a7bfb-original-pull-secret\") pod \"global-pull-secret-syncer-gwzk7\" (UID: \"348bacd3-9294-48e6-bbf4-60727b9a7bfb\") " pod="kube-system/global-pull-secret-syncer-gwzk7" Sep 10 03:48:59.418639 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:59.418618 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gwzk7" Sep 10 03:48:59.502956 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:59.502916 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhrqr\" (UniqueName: \"kubernetes.io/projected/10447409-1646-48fb-a5c9-494670b7d232-kube-api-access-rhrqr\") pod \"network-check-target-hlv2m\" (UID: \"10447409-1646-48fb-a5c9-494670b7d232\") " pod="openshift-network-diagnostics/network-check-target-hlv2m" Sep 10 03:48:59.505466 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:59.505444 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhrqr\" (UniqueName: \"kubernetes.io/projected/10447409-1646-48fb-a5c9-494670b7d232-kube-api-access-rhrqr\") pod \"network-check-target-hlv2m\" (UID: \"10447409-1646-48fb-a5c9-494670b7d232\") " pod="openshift-network-diagnostics/network-check-target-hlv2m" Sep 10 03:48:59.688814 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:59.688607 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gwzk7"] Sep 10 03:48:59.691161 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:48:59.691123 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod348bacd3_9294_48e6_bbf4_60727b9a7bfb.slice/crio-52a00b3c277b7abe7e3c1368e9998a2f0fcc2fd81f6c8b59a8a813ae540579ad WatchSource:0}: Error finding container 52a00b3c277b7abe7e3c1368e9998a2f0fcc2fd81f6c8b59a8a813ae540579ad: Status 404 returned error can't find the container with id 52a00b3c277b7abe7e3c1368e9998a2f0fcc2fd81f6c8b59a8a813ae540579ad Sep 10 03:48:59.729800 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:59.729774 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2jbt8\"" Sep 10 03:48:59.738616 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:59.738588 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hlv2m" Sep 10 03:48:59.880359 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:48:59.880314 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gwzk7" event={"ID":"348bacd3-9294-48e6-bbf4-60727b9a7bfb","Type":"ContainerStarted","Data":"52a00b3c277b7abe7e3c1368e9998a2f0fcc2fd81f6c8b59a8a813ae540579ad"} Sep 10 03:49:00.006978 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:49:00.006882 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hlv2m"] Sep 10 03:49:00.009941 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:49:00.009910 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10447409_1646_48fb_a5c9_494670b7d232.slice/crio-764e29ee7fb6a34f9c6eeaa037b344b5dd1e9052bd536990445342a7284f09b4 WatchSource:0}: Error finding container 764e29ee7fb6a34f9c6eeaa037b344b5dd1e9052bd536990445342a7284f09b4: Status 404 returned error can't find the container with id 764e29ee7fb6a34f9c6eeaa037b344b5dd1e9052bd536990445342a7284f09b4 Sep 10 03:49:00.885018 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:49:00.884978 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hlv2m" event={"ID":"10447409-1646-48fb-a5c9-494670b7d232","Type":"ContainerStarted","Data":"ab8abfe0cbe210d2cce1aaa3169b1596ead547fb8da6dce4e58f7c2bece917fe"} Sep 10 03:49:00.885478 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:49:00.885037 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hlv2m" event={"ID":"10447409-1646-48fb-a5c9-494670b7d232","Type":"ContainerStarted","Data":"764e29ee7fb6a34f9c6eeaa037b344b5dd1e9052bd536990445342a7284f09b4"} Sep 10 03:49:00.885478 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:49:00.885083 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-hlv2m" Sep 10 03:49:00.899992 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:49:00.899940 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-hlv2m" podStartSLOduration=66.899921558 podStartE2EDuration="1m6.899921558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:47:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:49:00.898834286 +0000 UTC m=+67.774897453" watchObservedRunningTime="2026-09-10 03:49:00.899921558 +0000 UTC m=+67.775984710" Sep 10 03:49:04.896543 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:49:04.896499 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gwzk7" event={"ID":"348bacd3-9294-48e6-bbf4-60727b9a7bfb","Type":"ContainerStarted","Data":"66859f8041987b893d5b2b7f0e0c4ee5ffe497c360b990a0f9263879f8b479d3"} Sep 10 03:49:04.914492 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:49:04.914436 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-gwzk7" podStartSLOduration=66.612745916 podStartE2EDuration="1m10.914419944s" totalImagesPullingTime="4.301674028s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 03:47:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:49:04.914045741 +0000 UTC m=+71.790108895" watchObservedRunningTime="2026-09-10 03:49:04.914419944 +0000 UTC m=+71.790483095" Sep 10 03:49:05.790119 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:49:05.790087 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:49:05.849472 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:49:05.849441 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:49:07.202430 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:49:07.202379 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:49:07.262650 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:49:07.262623 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:49:31.346574 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:49:31.346458 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-tls\") pod \"image-registry-554d6765f7-ss9hs\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:49:31.346574 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:49:31.346531 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/aa9e7159-181a-4a1f-afe1-33cd08e9ba6a-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-2mjrb\" (UID: \"aa9e7159-181a-4a1f-afe1-33cd08e9ba6a\") " pod="openshift-network-console/networking-console-plugin-79977f9844-2mjrb" Sep 10 03:49:31.347029 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:49:31.346640 2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 03:49:31.347029 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:49:31.346727 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa9e7159-181a-4a1f-afe1-33cd08e9ba6a-networking-console-plugin-cert podName:aa9e7159-181a-4a1f-afe1-33cd08e9ba6a nodeName:}" failed. No retries permitted until 2026-09-10 03:50:35.346709903 +0000 UTC m=+162.222773038 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/aa9e7159-181a-4a1f-afe1-33cd08e9ba6a-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-2mjrb" (UID: "aa9e7159-181a-4a1f-afe1-33cd08e9ba6a") : secret "networking-console-plugin-cert" not found Sep 10 03:49:31.347029 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:49:31.346643 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 03:49:31.347029 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:49:31.346756 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-554d6765f7-ss9hs: secret "image-registry-tls" not found Sep 10 03:49:31.347029 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:49:31.346806 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-tls podName:4ba38a7e-088d-4fdf-840c-c85b7826f1a6 nodeName:}" failed. No retries permitted until 2026-09-10 03:50:35.346793291 +0000 UTC m=+162.222856437 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-tls") pod "image-registry-554d6765f7-ss9hs" (UID: "4ba38a7e-088d-4fdf-840c-c85b7826f1a6") : secret "image-registry-tls" not found Sep 10 03:49:31.447375 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:49:31.447338 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/897fe719-a91a-4d75-8efb-4cb2fbade932-metrics-tls\") pod \"dns-default-dkkkk\" (UID: \"897fe719-a91a-4d75-8efb-4cb2fbade932\") " pod="openshift-dns/dns-default-dkkkk" Sep 10 03:49:31.447553 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:49:31.447500 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 03:49:31.447636 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:49:31.447623 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/897fe719-a91a-4d75-8efb-4cb2fbade932-metrics-tls podName:897fe719-a91a-4d75-8efb-4cb2fbade932 nodeName:}" failed. No retries permitted until 2026-09-10 03:50:35.447599376 +0000 UTC m=+162.323662525 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/897fe719-a91a-4d75-8efb-4cb2fbade932-metrics-tls") pod "dns-default-dkkkk" (UID: "897fe719-a91a-4d75-8efb-4cb2fbade932") : secret "dns-default-metrics-tls" not found Sep 10 03:49:31.890140 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:49:31.890109 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-hlv2m" Sep 10 03:49:42.213864 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:49:42.213826 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:49:42.268217 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:49:42.268182 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:50:03.485153 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:50:03.485093 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158bac16-c713-47cd-91d7-9fc36d96f570-metrics-certs\") pod \"network-metrics-daemon-2p9b8\" (UID: \"158bac16-c713-47cd-91d7-9fc36d96f570\") " pod="openshift-multus/network-metrics-daemon-2p9b8" Sep 10 03:50:03.485678 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:50:03.485248 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 10 03:50:03.485678 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:50:03.485334 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/158bac16-c713-47cd-91d7-9fc36d96f570-metrics-certs podName:158bac16-c713-47cd-91d7-9fc36d96f570 nodeName:}" failed. No retries permitted until 2026-09-10 03:52:05.485313698 +0000 UTC m=+252.361376839 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/158bac16-c713-47cd-91d7-9fc36d96f570-metrics-certs") pod "network-metrics-daemon-2p9b8" (UID: "158bac16-c713-47cd-91d7-9fc36d96f570") : secret "metrics-daemon-secret" not found Sep 10 03:50:30.462767 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:50:30.462717 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" podUID="4ba38a7e-088d-4fdf-840c-c85b7826f1a6" Sep 10 03:50:30.473075 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:50:30.473023 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-79977f9844-2mjrb" podUID="aa9e7159-181a-4a1f-afe1-33cd08e9ba6a" Sep 10 03:50:30.512396 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:50:30.512355 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-dkkkk" podUID="897fe719-a91a-4d75-8efb-4cb2fbade932" Sep 10 03:50:30.632445 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:50:30.632390 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-2p9b8" podUID="158bac16-c713-47cd-91d7-9fc36d96f570" Sep 10 03:50:31.111666 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:50:31.111633 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dkkkk" Sep 10 03:50:31.111822 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:50:31.111633 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:50:31.111822 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:50:31.111633 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-2mjrb" Sep 10 03:50:35.421671 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:50:35.421621 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-tls\") pod \"image-registry-554d6765f7-ss9hs\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:50:35.422078 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:50:35.421679 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/aa9e7159-181a-4a1f-afe1-33cd08e9ba6a-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-2mjrb\" (UID: \"aa9e7159-181a-4a1f-afe1-33cd08e9ba6a\") " pod="openshift-network-console/networking-console-plugin-79977f9844-2mjrb" Sep 10 03:50:35.422078 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:50:35.421779 2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 03:50:35.422078 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:50:35.421840 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa9e7159-181a-4a1f-afe1-33cd08e9ba6a-networking-console-plugin-cert podName:aa9e7159-181a-4a1f-afe1-33cd08e9ba6a nodeName:}" failed. No retries permitted until 2026-09-10 03:52:37.421826197 +0000 UTC m=+284.297889331 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/aa9e7159-181a-4a1f-afe1-33cd08e9ba6a-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-2mjrb" (UID: "aa9e7159-181a-4a1f-afe1-33cd08e9ba6a") : secret "networking-console-plugin-cert" not found Sep 10 03:50:35.422078 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:50:35.421783 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 03:50:35.422078 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:50:35.421872 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-554d6765f7-ss9hs: secret "image-registry-tls" not found Sep 10 03:50:35.422078 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:50:35.421924 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-tls podName:4ba38a7e-088d-4fdf-840c-c85b7826f1a6 nodeName:}" failed. No retries permitted until 2026-09-10 03:52:37.421910098 +0000 UTC m=+284.297973240 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-tls") pod "image-registry-554d6765f7-ss9hs" (UID: "4ba38a7e-088d-4fdf-840c-c85b7826f1a6") : secret "image-registry-tls" not found Sep 10 03:50:35.522927 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:50:35.522886 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/897fe719-a91a-4d75-8efb-4cb2fbade932-metrics-tls\") pod \"dns-default-dkkkk\" (UID: \"897fe719-a91a-4d75-8efb-4cb2fbade932\") " pod="openshift-dns/dns-default-dkkkk" Sep 10 03:50:35.523073 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:50:35.523036 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 03:50:35.523126 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:50:35.523113 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/897fe719-a91a-4d75-8efb-4cb2fbade932-metrics-tls podName:897fe719-a91a-4d75-8efb-4cb2fbade932 nodeName:}" failed. No retries permitted until 2026-09-10 03:52:37.523097524 +0000 UTC m=+284.399160653 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/897fe719-a91a-4d75-8efb-4cb2fbade932-metrics-tls") pod "dns-default-dkkkk" (UID: "897fe719-a91a-4d75-8efb-4cb2fbade932") : secret "dns-default-metrics-tls" not found Sep 10 03:50:43.599523 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:50:43.599484 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2p9b8" Sep 10 03:51:48.996601 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:51:48.996570 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:05.577058 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:05.576996 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158bac16-c713-47cd-91d7-9fc36d96f570-metrics-certs\") pod \"network-metrics-daemon-2p9b8\" (UID: \"158bac16-c713-47cd-91d7-9fc36d96f570\") " pod="openshift-multus/network-metrics-daemon-2p9b8" Sep 10 03:52:05.577539 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:05.577152 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 10 03:52:05.577539 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:05.577227 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/158bac16-c713-47cd-91d7-9fc36d96f570-metrics-certs podName:158bac16-c713-47cd-91d7-9fc36d96f570 nodeName:}" failed. No retries permitted until 2026-09-10 03:54:07.57721232 +0000 UTC m=+374.453275455 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/158bac16-c713-47cd-91d7-9fc36d96f570-metrics-certs") pod "network-metrics-daemon-2p9b8" (UID: "158bac16-c713-47cd-91d7-9fc36d96f570") : secret "metrics-daemon-secret" not found Sep 10 03:52:15.192959 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.192915 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-q5p2x"] Sep 10 03:52:15.196033 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.196010 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-q5p2x" Sep 10 03:52:15.197667 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.197643 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jlq9n"] Sep 10 03:52:15.200325 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.200307 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-hf7wl"] Sep 10 03:52:15.200406 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.200390 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jlq9n" Sep 10 03:52:15.203171 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.203151 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-hf7wl" Sep 10 03:52:15.204010 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.203991 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 10 03:52:15.204010 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.203999 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-6tlwk\"" Sep 10 03:52:15.204455 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.204440 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 10 03:52:15.204520 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.204490 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-7x9m7\"" Sep 10 03:52:15.204568 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.204525 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Sep 10 03:52:15.205829 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.205810 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Sep 10 03:52:15.205829 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.205836 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Sep 10 03:52:15.205982 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.205844 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Sep 10 03:52:15.205982 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.205817 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Sep 10 03:52:15.205982 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.205894 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Sep 10 03:52:15.206832 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.206803 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-mgv55\"" Sep 10 03:52:15.206832 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.206813 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Sep 10 03:52:15.206980 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.206864 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Sep 10 03:52:15.206980 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.206923 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Sep 10 03:52:15.211897 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.211878 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Sep 10 03:52:15.220399 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.220371 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-q5p2x"] Sep 10 03:52:15.221574 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.221549 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jlq9n"] Sep 10 03:52:15.234127 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.234096 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-hf7wl"] Sep 10 03:52:15.244255 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.244225 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/678609c3-b3fa-4beb-953d-d61067e32212-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-q5p2x\" (UID: \"678609c3-b3fa-4beb-953d-d61067e32212\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-q5p2x" Sep 10 03:52:15.244255 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.244258 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbpc7\" (UniqueName: \"kubernetes.io/projected/50451d7c-07fb-4641-a36f-d142e72b5e81-kube-api-access-nbpc7\") pod \"console-operator-85b454874b-hf7wl\" (UID: \"50451d7c-07fb-4641-a36f-d142e72b5e81\") " pod="openshift-console-operator/console-operator-85b454874b-hf7wl" Sep 10 03:52:15.244495 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.244315 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3b13de7-3c45-4da0-b80d-06508774147c-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-jlq9n\" (UID: \"b3b13de7-3c45-4da0-b80d-06508774147c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jlq9n" Sep 10 03:52:15.244495 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.244408 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v855m\" (UniqueName: \"kubernetes.io/projected/678609c3-b3fa-4beb-953d-d61067e32212-kube-api-access-v855m\") pod \"cluster-monitoring-operator-6cdcc96965-q5p2x\" (UID: \"678609c3-b3fa-4beb-953d-d61067e32212\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-q5p2x" Sep 10 03:52:15.244495 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.244479 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/678609c3-b3fa-4beb-953d-d61067e32212-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-q5p2x\" (UID: \"678609c3-b3fa-4beb-953d-d61067e32212\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-q5p2x" Sep 10 03:52:15.244604 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.244522 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlfhl\" (UniqueName: \"kubernetes.io/projected/b3b13de7-3c45-4da0-b80d-06508774147c-kube-api-access-jlfhl\") pod \"cluster-samples-operator-7cdcfcf76c-jlq9n\" (UID: \"b3b13de7-3c45-4da0-b80d-06508774147c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jlq9n" Sep 10 03:52:15.244604 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.244563 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/50451d7c-07fb-4641-a36f-d142e72b5e81-config\") pod \"console-operator-85b454874b-hf7wl\" (UID: \"50451d7c-07fb-4641-a36f-d142e72b5e81\") " pod="openshift-console-operator/console-operator-85b454874b-hf7wl" Sep 10 03:52:15.244604 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.244578 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/50451d7c-07fb-4641-a36f-d142e72b5e81-serving-cert\") pod \"console-operator-85b454874b-hf7wl\" (UID: \"50451d7c-07fb-4641-a36f-d142e72b5e81\") " pod="openshift-console-operator/console-operator-85b454874b-hf7wl" Sep 10 03:52:15.244604 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.244592 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/50451d7c-07fb-4641-a36f-d142e72b5e81-trusted-ca\") pod \"console-operator-85b454874b-hf7wl\" (UID: \"50451d7c-07fb-4641-a36f-d142e72b5e81\") " pod="openshift-console-operator/console-operator-85b454874b-hf7wl" Sep 10 03:52:15.292104 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.292068 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-2zth2"] Sep 10 03:52:15.295052 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.295035 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-2zth2" Sep 10 03:52:15.297288 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.297260 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Sep 10 03:52:15.297426 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.297326 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Sep 10 03:52:15.297661 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.297641 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-w8xfq\"" Sep 10 03:52:15.305143 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.305118 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-2zth2"] Sep 10 03:52:15.344362 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.344328 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:15.345308 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.345274 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/678609c3-b3fa-4beb-953d-d61067e32212-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-q5p2x\" (UID: \"678609c3-b3fa-4beb-953d-d61067e32212\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-q5p2x" Sep 10 03:52:15.345378 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.345333 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlfhl\" (UniqueName: \"kubernetes.io/projected/b3b13de7-3c45-4da0-b80d-06508774147c-kube-api-access-jlfhl\") pod \"cluster-samples-operator-7cdcfcf76c-jlq9n\" (UID: \"b3b13de7-3c45-4da0-b80d-06508774147c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jlq9n" Sep 10 03:52:15.345378 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.345354 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88r9g\" (UniqueName: \"kubernetes.io/projected/881d05c0-8849-4caa-98fb-4a41ce400d67-kube-api-access-88r9g\") pod \"volume-data-source-validator-695c895bd6-2zth2\" (UID: \"881d05c0-8849-4caa-98fb-4a41ce400d67\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-2zth2" Sep 10 03:52:15.345483 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.345381 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/50451d7c-07fb-4641-a36f-d142e72b5e81-config\") pod \"console-operator-85b454874b-hf7wl\" (UID: \"50451d7c-07fb-4641-a36f-d142e72b5e81\") " pod="openshift-console-operator/console-operator-85b454874b-hf7wl" Sep 10 03:52:15.345483 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.345395 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/50451d7c-07fb-4641-a36f-d142e72b5e81-serving-cert\") pod \"console-operator-85b454874b-hf7wl\" (UID: \"50451d7c-07fb-4641-a36f-d142e72b5e81\") " pod="openshift-console-operator/console-operator-85b454874b-hf7wl" Sep 10 03:52:15.345483 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.345409 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/50451d7c-07fb-4641-a36f-d142e72b5e81-trusted-ca\") pod \"console-operator-85b454874b-hf7wl\" (UID: \"50451d7c-07fb-4641-a36f-d142e72b5e81\") " pod="openshift-console-operator/console-operator-85b454874b-hf7wl" Sep 10 03:52:15.345483 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:15.345446 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 03:52:15.345662 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:15.345514 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/678609c3-b3fa-4beb-953d-d61067e32212-cluster-monitoring-operator-tls podName:678609c3-b3fa-4beb-953d-d61067e32212 nodeName:}" failed. No retries permitted until 2026-09-10 03:52:15.845494645 +0000 UTC m=+262.721557777 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/678609c3-b3fa-4beb-953d-d61067e32212-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-q5p2x" (UID: "678609c3-b3fa-4beb-953d-d61067e32212") : secret "cluster-monitoring-operator-tls" not found Sep 10 03:52:15.345662 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.345449 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/678609c3-b3fa-4beb-953d-d61067e32212-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-q5p2x\" (UID: \"678609c3-b3fa-4beb-953d-d61067e32212\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-q5p2x" Sep 10 03:52:15.345662 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.345566 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbpc7\" (UniqueName: \"kubernetes.io/projected/50451d7c-07fb-4641-a36f-d142e72b5e81-kube-api-access-nbpc7\") pod \"console-operator-85b454874b-hf7wl\" (UID: \"50451d7c-07fb-4641-a36f-d142e72b5e81\") " pod="openshift-console-operator/console-operator-85b454874b-hf7wl" Sep 10 03:52:15.345662 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.345621 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3b13de7-3c45-4da0-b80d-06508774147c-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-jlq9n\" (UID: \"b3b13de7-3c45-4da0-b80d-06508774147c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jlq9n" Sep 10 03:52:15.345867 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.345665 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v855m\" (UniqueName: \"kubernetes.io/projected/678609c3-b3fa-4beb-953d-d61067e32212-kube-api-access-v855m\") pod \"cluster-monitoring-operator-6cdcc96965-q5p2x\" (UID: \"678609c3-b3fa-4beb-953d-d61067e32212\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-q5p2x" Sep 10 03:52:15.345867 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:15.345806 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 03:52:15.345867 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:15.345852 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3b13de7-3c45-4da0-b80d-06508774147c-samples-operator-tls podName:b3b13de7-3c45-4da0-b80d-06508774147c nodeName:}" failed. No retries permitted until 2026-09-10 03:52:15.845837176 +0000 UTC m=+262.721900311 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3b13de7-3c45-4da0-b80d-06508774147c-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-jlq9n" (UID: "b3b13de7-3c45-4da0-b80d-06508774147c") : secret "samples-operator-tls" not found Sep 10 03:52:15.346230 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.346201 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/678609c3-b3fa-4beb-953d-d61067e32212-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-q5p2x\" (UID: \"678609c3-b3fa-4beb-953d-d61067e32212\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-q5p2x" Sep 10 03:52:15.346362 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.346341 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/50451d7c-07fb-4641-a36f-d142e72b5e81-trusted-ca\") pod \"console-operator-85b454874b-hf7wl\" (UID: \"50451d7c-07fb-4641-a36f-d142e72b5e81\") " pod="openshift-console-operator/console-operator-85b454874b-hf7wl" Sep 10 03:52:15.346481 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.346467 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/50451d7c-07fb-4641-a36f-d142e72b5e81-config\") pod \"console-operator-85b454874b-hf7wl\" (UID: \"50451d7c-07fb-4641-a36f-d142e72b5e81\") " pod="openshift-console-operator/console-operator-85b454874b-hf7wl" Sep 10 03:52:15.347877 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.347848 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/50451d7c-07fb-4641-a36f-d142e72b5e81-serving-cert\") pod \"console-operator-85b454874b-hf7wl\" (UID: \"50451d7c-07fb-4641-a36f-d142e72b5e81\") " pod="openshift-console-operator/console-operator-85b454874b-hf7wl" Sep 10 03:52:15.355447 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.355423 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbpc7\" (UniqueName: \"kubernetes.io/projected/50451d7c-07fb-4641-a36f-d142e72b5e81-kube-api-access-nbpc7\") pod \"console-operator-85b454874b-hf7wl\" (UID: \"50451d7c-07fb-4641-a36f-d142e72b5e81\") " pod="openshift-console-operator/console-operator-85b454874b-hf7wl" Sep 10 03:52:15.355548 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.355449 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v855m\" (UniqueName: \"kubernetes.io/projected/678609c3-b3fa-4beb-953d-d61067e32212-kube-api-access-v855m\") pod \"cluster-monitoring-operator-6cdcc96965-q5p2x\" (UID: \"678609c3-b3fa-4beb-953d-d61067e32212\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-q5p2x" Sep 10 03:52:15.355548 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.355496 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlfhl\" (UniqueName: \"kubernetes.io/projected/b3b13de7-3c45-4da0-b80d-06508774147c-kube-api-access-jlfhl\") pod \"cluster-samples-operator-7cdcfcf76c-jlq9n\" (UID: \"b3b13de7-3c45-4da0-b80d-06508774147c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jlq9n" Sep 10 03:52:15.398091 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.398053 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:15.446375 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.446275 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88r9g\" (UniqueName: \"kubernetes.io/projected/881d05c0-8849-4caa-98fb-4a41ce400d67-kube-api-access-88r9g\") pod \"volume-data-source-validator-695c895bd6-2zth2\" (UID: \"881d05c0-8849-4caa-98fb-4a41ce400d67\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-2zth2" Sep 10 03:52:15.454470 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.454433 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88r9g\" (UniqueName: \"kubernetes.io/projected/881d05c0-8849-4caa-98fb-4a41ce400d67-kube-api-access-88r9g\") pod \"volume-data-source-validator-695c895bd6-2zth2\" (UID: \"881d05c0-8849-4caa-98fb-4a41ce400d67\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-2zth2" Sep 10 03:52:15.518902 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.518843 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-hf7wl" Sep 10 03:52:15.603893 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.603863 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-2zth2" Sep 10 03:52:15.785621 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.785530 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-hf7wl"] Sep 10 03:52:15.789555 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:52:15.789516 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50451d7c_07fb_4641_a36f_d142e72b5e81.slice/crio-6ff33e52677651ffe09fe486b1c3fd82a73a745ff4b339e604bf19f04154a3ab WatchSource:0}: Error finding container 6ff33e52677651ffe09fe486b1c3fd82a73a745ff4b339e604bf19f04154a3ab: Status 404 returned error can't find the container with id 6ff33e52677651ffe09fe486b1c3fd82a73a745ff4b339e604bf19f04154a3ab Sep 10 03:52:15.850008 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.849919 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3b13de7-3c45-4da0-b80d-06508774147c-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-jlq9n\" (UID: \"b3b13de7-3c45-4da0-b80d-06508774147c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jlq9n" Sep 10 03:52:15.850008 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.849978 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/678609c3-b3fa-4beb-953d-d61067e32212-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-q5p2x\" (UID: \"678609c3-b3fa-4beb-953d-d61067e32212\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-q5p2x" Sep 10 03:52:15.850201 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:15.850086 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 03:52:15.850201 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:15.850165 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3b13de7-3c45-4da0-b80d-06508774147c-samples-operator-tls podName:b3b13de7-3c45-4da0-b80d-06508774147c nodeName:}" failed. No retries permitted until 2026-09-10 03:52:16.85014565 +0000 UTC m=+263.726208779 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3b13de7-3c45-4da0-b80d-06508774147c-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-jlq9n" (UID: "b3b13de7-3c45-4da0-b80d-06508774147c") : secret "samples-operator-tls" not found Sep 10 03:52:15.850279 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:15.850215 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 03:52:15.850279 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:15.850264 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/678609c3-b3fa-4beb-953d-d61067e32212-cluster-monitoring-operator-tls podName:678609c3-b3fa-4beb-953d-d61067e32212 nodeName:}" failed. No retries permitted until 2026-09-10 03:52:16.850252394 +0000 UTC m=+263.726315523 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/678609c3-b3fa-4beb-953d-d61067e32212-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-q5p2x" (UID: "678609c3-b3fa-4beb-953d-d61067e32212") : secret "cluster-monitoring-operator-tls" not found Sep 10 03:52:15.869883 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:15.869844 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-2zth2"] Sep 10 03:52:15.876052 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:52:15.876021 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod881d05c0_8849_4caa_98fb_4a41ce400d67.slice/crio-b85a5c0a22f91a144cba2900dc375787dda8eaedecc14df495234205a41aff7e WatchSource:0}: Error finding container b85a5c0a22f91a144cba2900dc375787dda8eaedecc14df495234205a41aff7e: Status 404 returned error can't find the container with id b85a5c0a22f91a144cba2900dc375787dda8eaedecc14df495234205a41aff7e Sep 10 03:52:16.366440 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:16.366397 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-2zth2" event={"ID":"881d05c0-8849-4caa-98fb-4a41ce400d67","Type":"ContainerStarted","Data":"b85a5c0a22f91a144cba2900dc375787dda8eaedecc14df495234205a41aff7e"} Sep 10 03:52:16.367826 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:16.367797 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-hf7wl" event={"ID":"50451d7c-07fb-4641-a36f-d142e72b5e81","Type":"ContainerStarted","Data":"6ff33e52677651ffe09fe486b1c3fd82a73a745ff4b339e604bf19f04154a3ab"} Sep 10 03:52:16.859042 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:16.858994 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3b13de7-3c45-4da0-b80d-06508774147c-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-jlq9n\" (UID: \"b3b13de7-3c45-4da0-b80d-06508774147c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jlq9n" Sep 10 03:52:16.859245 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:16.859076 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/678609c3-b3fa-4beb-953d-d61067e32212-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-q5p2x\" (UID: \"678609c3-b3fa-4beb-953d-d61067e32212\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-q5p2x" Sep 10 03:52:16.859245 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:16.859176 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 03:52:16.859245 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:16.859223 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 03:52:16.859465 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:16.859270 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3b13de7-3c45-4da0-b80d-06508774147c-samples-operator-tls podName:b3b13de7-3c45-4da0-b80d-06508774147c nodeName:}" failed. No retries permitted until 2026-09-10 03:52:18.859247022 +0000 UTC m=+265.735310163 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3b13de7-3c45-4da0-b80d-06508774147c-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-jlq9n" (UID: "b3b13de7-3c45-4da0-b80d-06508774147c") : secret "samples-operator-tls" not found Sep 10 03:52:16.859465 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:16.859308 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/678609c3-b3fa-4beb-953d-d61067e32212-cluster-monitoring-operator-tls podName:678609c3-b3fa-4beb-953d-d61067e32212 nodeName:}" failed. No retries permitted until 2026-09-10 03:52:18.859282054 +0000 UTC m=+265.735345191 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/678609c3-b3fa-4beb-953d-d61067e32212-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-q5p2x" (UID: "678609c3-b3fa-4beb-953d-d61067e32212") : secret "cluster-monitoring-operator-tls" not found Sep 10 03:52:17.373455 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:17.373366 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-2zth2" event={"ID":"881d05c0-8849-4caa-98fb-4a41ce400d67","Type":"ContainerStarted","Data":"a953cb19efa164dd5c6fc140df0a3e4b88d2615b315a55613a824077ebf74af7"} Sep 10 03:52:17.387798 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:17.387726 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-2zth2" podStartSLOduration=1.214217295 podStartE2EDuration="2.38770592s" totalImagesPullingTime="1.173488625s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:52:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:52:17.386864148 +0000 UTC m=+264.262927299" watchObservedRunningTime="2026-09-10 03:52:17.38770592 +0000 UTC m=+264.263769073" Sep 10 03:52:17.464976 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:17.464937 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:17.521203 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:17.521163 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:18.378841 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:18.378817 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hf7wl_50451d7c-07fb-4641-a36f-d142e72b5e81/console-operator/0.log" Sep 10 03:52:18.379198 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:18.378856 2562 generic.go:356] "Generic (PLEG): container finished" podID="50451d7c-07fb-4641-a36f-d142e72b5e81" containerID="f1d9d46a65c08f0b573e019c510fb3651a818eb21de146d799b4d96b937aec3b" exitCode=255 Sep 10 03:52:18.379198 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:18.378889 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-hf7wl" event={"ID":"50451d7c-07fb-4641-a36f-d142e72b5e81","Type":"ContainerDied","Data":"f1d9d46a65c08f0b573e019c510fb3651a818eb21de146d799b4d96b937aec3b"} Sep 10 03:52:18.379273 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:18.379230 2562 scope.go:117] "RemoveContainer" containerID="f1d9d46a65c08f0b573e019c510fb3651a818eb21de146d799b4d96b937aec3b" Sep 10 03:52:18.878654 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:18.878594 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/678609c3-b3fa-4beb-953d-d61067e32212-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-q5p2x\" (UID: \"678609c3-b3fa-4beb-953d-d61067e32212\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-q5p2x" Sep 10 03:52:18.878843 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:18.878717 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3b13de7-3c45-4da0-b80d-06508774147c-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-jlq9n\" (UID: \"b3b13de7-3c45-4da0-b80d-06508774147c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jlq9n" Sep 10 03:52:18.878843 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:18.878754 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 03:52:18.878843 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:18.878824 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 03:52:18.878940 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:18.878826 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/678609c3-b3fa-4beb-953d-d61067e32212-cluster-monitoring-operator-tls podName:678609c3-b3fa-4beb-953d-d61067e32212 nodeName:}" failed. No retries permitted until 2026-09-10 03:52:22.87880955 +0000 UTC m=+269.754872680 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/678609c3-b3fa-4beb-953d-d61067e32212-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-q5p2x" (UID: "678609c3-b3fa-4beb-953d-d61067e32212") : secret "cluster-monitoring-operator-tls" not found Sep 10 03:52:18.878940 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:18.878878 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3b13de7-3c45-4da0-b80d-06508774147c-samples-operator-tls podName:b3b13de7-3c45-4da0-b80d-06508774147c nodeName:}" failed. No retries permitted until 2026-09-10 03:52:22.878861954 +0000 UTC m=+269.754925084 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3b13de7-3c45-4da0-b80d-06508774147c-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-jlq9n" (UID: "b3b13de7-3c45-4da0-b80d-06508774147c") : secret "samples-operator-tls" not found Sep 10 03:52:19.383110 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:19.383083 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hf7wl_50451d7c-07fb-4641-a36f-d142e72b5e81/console-operator/1.log" Sep 10 03:52:19.383599 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:19.383531 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hf7wl_50451d7c-07fb-4641-a36f-d142e72b5e81/console-operator/0.log" Sep 10 03:52:19.383599 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:19.383575 2562 generic.go:356] "Generic (PLEG): container finished" podID="50451d7c-07fb-4641-a36f-d142e72b5e81" containerID="b0a70a1556f90bdabaf180edf973a9895b85dc9b60378441d1da37245c2c1416" exitCode=255 Sep 10 03:52:19.383712 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:19.383636 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-hf7wl" event={"ID":"50451d7c-07fb-4641-a36f-d142e72b5e81","Type":"ContainerDied","Data":"b0a70a1556f90bdabaf180edf973a9895b85dc9b60378441d1da37245c2c1416"} Sep 10 03:52:19.383712 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:19.383689 2562 scope.go:117] "RemoveContainer" containerID="f1d9d46a65c08f0b573e019c510fb3651a818eb21de146d799b4d96b937aec3b" Sep 10 03:52:19.383889 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:19.383871 2562 scope.go:117] "RemoveContainer" containerID="b0a70a1556f90bdabaf180edf973a9895b85dc9b60378441d1da37245c2c1416" Sep 10 03:52:19.384086 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:19.384067 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-hf7wl_openshift-console-operator(50451d7c-07fb-4641-a36f-d142e72b5e81)\"" pod="openshift-console-operator/console-operator-85b454874b-hf7wl" podUID="50451d7c-07fb-4641-a36f-d142e72b5e81" Sep 10 03:52:20.104576 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:20.104545 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:20.220227 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:20.220197 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:20.237699 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:20.237668 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-54dww"] Sep 10 03:52:20.242077 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:20.242056 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-54dww" Sep 10 03:52:20.244009 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:20.243979 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 10 03:52:20.244149 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:20.244083 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 10 03:52:20.244237 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:20.244218 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 10 03:52:20.244835 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:20.244816 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 10 03:52:20.244916 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:20.244822 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nhqj9\"" Sep 10 03:52:20.254630 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:20.254607 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-54dww"] Sep 10 03:52:20.289198 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:20.289161 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:20.387567 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:20.387489 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hf7wl_50451d7c-07fb-4641-a36f-d142e72b5e81/console-operator/1.log" Sep 10 03:52:20.387938 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:20.387813 2562 scope.go:117] "RemoveContainer" containerID="b0a70a1556f90bdabaf180edf973a9895b85dc9b60378441d1da37245c2c1416" Sep 10 03:52:20.387998 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:20.387977 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-hf7wl_openshift-console-operator(50451d7c-07fb-4641-a36f-d142e72b5e81)\"" pod="openshift-console-operator/console-operator-85b454874b-hf7wl" podUID="50451d7c-07fb-4641-a36f-d142e72b5e81" Sep 10 03:52:20.389074 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:20.389052 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqwzd\" (UniqueName: \"kubernetes.io/projected/74b614da-e905-4ff7-9bfe-2d9bea24859b-kube-api-access-wqwzd\") pod \"insights-runtime-extractor-54dww\" (UID: \"74b614da-e905-4ff7-9bfe-2d9bea24859b\") " pod="openshift-insights/insights-runtime-extractor-54dww" Sep 10 03:52:20.389181 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:20.389115 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/74b614da-e905-4ff7-9bfe-2d9bea24859b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-54dww\" (UID: \"74b614da-e905-4ff7-9bfe-2d9bea24859b\") " pod="openshift-insights/insights-runtime-extractor-54dww" Sep 10 03:52:20.389220 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:20.389176 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/74b614da-e905-4ff7-9bfe-2d9bea24859b-data-volume\") pod \"insights-runtime-extractor-54dww\" (UID: \"74b614da-e905-4ff7-9bfe-2d9bea24859b\") " pod="openshift-insights/insights-runtime-extractor-54dww" Sep 10 03:52:20.389220 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:20.389209 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/74b614da-e905-4ff7-9bfe-2d9bea24859b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-54dww\" (UID: \"74b614da-e905-4ff7-9bfe-2d9bea24859b\") " pod="openshift-insights/insights-runtime-extractor-54dww" Sep 10 03:52:20.389317 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:20.389234 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/74b614da-e905-4ff7-9bfe-2d9bea24859b-crio-socket\") pod \"insights-runtime-extractor-54dww\" (UID: \"74b614da-e905-4ff7-9bfe-2d9bea24859b\") " pod="openshift-insights/insights-runtime-extractor-54dww" Sep 10 03:52:20.490394 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:20.490344 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/74b614da-e905-4ff7-9bfe-2d9bea24859b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-54dww\" (UID: \"74b614da-e905-4ff7-9bfe-2d9bea24859b\") " pod="openshift-insights/insights-runtime-extractor-54dww" Sep 10 03:52:20.490394 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:20.490400 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/74b614da-e905-4ff7-9bfe-2d9bea24859b-crio-socket\") pod \"insights-runtime-extractor-54dww\" (UID: \"74b614da-e905-4ff7-9bfe-2d9bea24859b\") " pod="openshift-insights/insights-runtime-extractor-54dww" Sep 10 03:52:20.490648 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:20.490470 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 03:52:20.490648 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:20.490541 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/74b614da-e905-4ff7-9bfe-2d9bea24859b-kube-rbac-proxy-cm podName:74b614da-e905-4ff7-9bfe-2d9bea24859b nodeName:}" failed. No retries permitted until 2026-09-10 03:52:20.990524339 +0000 UTC m=+267.866587489 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/74b614da-e905-4ff7-9bfe-2d9bea24859b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-54dww" (UID: "74b614da-e905-4ff7-9bfe-2d9bea24859b") : configmap "kube-rbac-proxy" not found Sep 10 03:52:20.490648 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:20.490539 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/74b614da-e905-4ff7-9bfe-2d9bea24859b-crio-socket\") pod \"insights-runtime-extractor-54dww\" (UID: \"74b614da-e905-4ff7-9bfe-2d9bea24859b\") " pod="openshift-insights/insights-runtime-extractor-54dww" Sep 10 03:52:20.490648 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:20.490472 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqwzd\" (UniqueName: \"kubernetes.io/projected/74b614da-e905-4ff7-9bfe-2d9bea24859b-kube-api-access-wqwzd\") pod \"insights-runtime-extractor-54dww\" (UID: \"74b614da-e905-4ff7-9bfe-2d9bea24859b\") " pod="openshift-insights/insights-runtime-extractor-54dww" Sep 10 03:52:20.490854 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:20.490650 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/74b614da-e905-4ff7-9bfe-2d9bea24859b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-54dww\" (UID: \"74b614da-e905-4ff7-9bfe-2d9bea24859b\") " pod="openshift-insights/insights-runtime-extractor-54dww" Sep 10 03:52:20.490854 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:20.490724 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/74b614da-e905-4ff7-9bfe-2d9bea24859b-data-volume\") pod \"insights-runtime-extractor-54dww\" (UID: \"74b614da-e905-4ff7-9bfe-2d9bea24859b\") " pod="openshift-insights/insights-runtime-extractor-54dww" Sep 10 03:52:20.490854 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:20.490839 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 03:52:20.491008 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:20.490912 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74b614da-e905-4ff7-9bfe-2d9bea24859b-insights-runtime-extractor-tls podName:74b614da-e905-4ff7-9bfe-2d9bea24859b nodeName:}" failed. No retries permitted until 2026-09-10 03:52:20.990893019 +0000 UTC m=+267.866956150 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/74b614da-e905-4ff7-9bfe-2d9bea24859b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-54dww" (UID: "74b614da-e905-4ff7-9bfe-2d9bea24859b") : secret "insights-runtime-extractor-tls" not found Sep 10 03:52:20.491066 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:20.491042 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/74b614da-e905-4ff7-9bfe-2d9bea24859b-data-volume\") pod \"insights-runtime-extractor-54dww\" (UID: \"74b614da-e905-4ff7-9bfe-2d9bea24859b\") " pod="openshift-insights/insights-runtime-extractor-54dww" Sep 10 03:52:20.498448 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:20.498422 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqwzd\" (UniqueName: \"kubernetes.io/projected/74b614da-e905-4ff7-9bfe-2d9bea24859b-kube-api-access-wqwzd\") pod \"insights-runtime-extractor-54dww\" (UID: \"74b614da-e905-4ff7-9bfe-2d9bea24859b\") " pod="openshift-insights/insights-runtime-extractor-54dww" Sep 10 03:52:20.535480 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:20.535441 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:20.810449 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:20.810415 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:20.909596 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:20.909561 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:20.995406 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:20.995374 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/74b614da-e905-4ff7-9bfe-2d9bea24859b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-54dww\" (UID: \"74b614da-e905-4ff7-9bfe-2d9bea24859b\") " pod="openshift-insights/insights-runtime-extractor-54dww" Sep 10 03:52:20.995516 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:20.995466 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/74b614da-e905-4ff7-9bfe-2d9bea24859b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-54dww\" (UID: \"74b614da-e905-4ff7-9bfe-2d9bea24859b\") " pod="openshift-insights/insights-runtime-extractor-54dww" Sep 10 03:52:20.995516 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:20.995501 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 03:52:20.995582 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:20.995566 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 03:52:20.995616 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:20.995586 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/74b614da-e905-4ff7-9bfe-2d9bea24859b-kube-rbac-proxy-cm podName:74b614da-e905-4ff7-9bfe-2d9bea24859b nodeName:}" failed. No retries permitted until 2026-09-10 03:52:21.99556763 +0000 UTC m=+268.871630767 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/74b614da-e905-4ff7-9bfe-2d9bea24859b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-54dww" (UID: "74b614da-e905-4ff7-9bfe-2d9bea24859b") : configmap "kube-rbac-proxy" not found Sep 10 03:52:20.995616 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:20.995606 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74b614da-e905-4ff7-9bfe-2d9bea24859b-insights-runtime-extractor-tls podName:74b614da-e905-4ff7-9bfe-2d9bea24859b nodeName:}" failed. No retries permitted until 2026-09-10 03:52:21.995595608 +0000 UTC m=+268.871658738 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/74b614da-e905-4ff7-9bfe-2d9bea24859b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-54dww" (UID: "74b614da-e905-4ff7-9bfe-2d9bea24859b") : secret "insights-runtime-extractor-tls" not found Sep 10 03:52:21.212488 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:21.212453 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:21.248714 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:21.248676 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-gz76m"] Sep 10 03:52:21.251885 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:21.251867 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-gz76m" Sep 10 03:52:21.254011 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:21.253981 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Sep 10 03:52:21.254146 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:21.254105 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-6btlh\"" Sep 10 03:52:21.254846 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:21.254822 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Sep 10 03:52:21.260639 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:21.260613 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-gz76m"] Sep 10 03:52:21.275893 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:21.275865 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:21.326893 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:21.326861 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:21.398979 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:21.398940 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9mt9\" (UniqueName: \"kubernetes.io/projected/4f351318-0f05-4b9c-9266-73b5ca137fa8-kube-api-access-d9mt9\") pod \"migrator-fdbb8fdff-gz76m\" (UID: \"4f351318-0f05-4b9c-9266-73b5ca137fa8\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-gz76m" Sep 10 03:52:21.500363 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:21.500268 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9mt9\" (UniqueName: \"kubernetes.io/projected/4f351318-0f05-4b9c-9266-73b5ca137fa8-kube-api-access-d9mt9\") pod \"migrator-fdbb8fdff-gz76m\" (UID: \"4f351318-0f05-4b9c-9266-73b5ca137fa8\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-gz76m" Sep 10 03:52:21.507701 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:21.507668 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9mt9\" (UniqueName: \"kubernetes.io/projected/4f351318-0f05-4b9c-9266-73b5ca137fa8-kube-api-access-d9mt9\") pod \"migrator-fdbb8fdff-gz76m\" (UID: \"4f351318-0f05-4b9c-9266-73b5ca137fa8\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-gz76m" Sep 10 03:52:21.561991 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:21.561949 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-gz76m" Sep 10 03:52:21.831816 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:21.831731 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-gz76m"] Sep 10 03:52:21.834821 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:52:21.834790 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f351318_0f05_4b9c_9266_73b5ca137fa8.slice/crio-ae82d2b715b7fade8a0a412d1098943cb2068acaeed3792b5a87efb4d15af0c7 WatchSource:0}: Error finding container ae82d2b715b7fade8a0a412d1098943cb2068acaeed3792b5a87efb4d15af0c7: Status 404 returned error can't find the container with id ae82d2b715b7fade8a0a412d1098943cb2068acaeed3792b5a87efb4d15af0c7 Sep 10 03:52:22.004651 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:22.004615 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/74b614da-e905-4ff7-9bfe-2d9bea24859b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-54dww\" (UID: \"74b614da-e905-4ff7-9bfe-2d9bea24859b\") " pod="openshift-insights/insights-runtime-extractor-54dww" Sep 10 03:52:22.004869 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:22.004743 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 03:52:22.004869 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:22.004746 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/74b614da-e905-4ff7-9bfe-2d9bea24859b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-54dww\" (UID: \"74b614da-e905-4ff7-9bfe-2d9bea24859b\") " pod="openshift-insights/insights-runtime-extractor-54dww" Sep 10 03:52:22.004869 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:22.004812 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/74b614da-e905-4ff7-9bfe-2d9bea24859b-kube-rbac-proxy-cm podName:74b614da-e905-4ff7-9bfe-2d9bea24859b nodeName:}" failed. No retries permitted until 2026-09-10 03:52:24.004794117 +0000 UTC m=+270.880857252 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/74b614da-e905-4ff7-9bfe-2d9bea24859b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-54dww" (UID: "74b614da-e905-4ff7-9bfe-2d9bea24859b") : configmap "kube-rbac-proxy" not found Sep 10 03:52:22.004869 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:22.004841 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 03:52:22.005038 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:22.004904 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74b614da-e905-4ff7-9bfe-2d9bea24859b-insights-runtime-extractor-tls podName:74b614da-e905-4ff7-9bfe-2d9bea24859b nodeName:}" failed. No retries permitted until 2026-09-10 03:52:24.004889124 +0000 UTC m=+270.880952267 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/74b614da-e905-4ff7-9bfe-2d9bea24859b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-54dww" (UID: "74b614da-e905-4ff7-9bfe-2d9bea24859b") : secret "insights-runtime-extractor-tls" not found Sep 10 03:52:22.395453 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:22.395421 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-gz76m" event={"ID":"4f351318-0f05-4b9c-9266-73b5ca137fa8","Type":"ContainerStarted","Data":"ae82d2b715b7fade8a0a412d1098943cb2068acaeed3792b5a87efb4d15af0c7"} Sep 10 03:52:22.409226 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:22.409196 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fwlqw_460613b7-2cdf-4e57-8edf-06e7b72f790a/dns-node-resolver/0.log" Sep 10 03:52:22.409611 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:22.409378 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-75d99c484-8c28k"] Sep 10 03:52:22.413479 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:22.413459 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-8c28k" Sep 10 03:52:22.415800 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:22.415774 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Sep 10 03:52:22.415800 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:22.415788 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Sep 10 03:52:22.416526 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:22.416510 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-qbjxm\"" Sep 10 03:52:22.416608 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:22.416561 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Sep 10 03:52:22.416654 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:22.416612 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Sep 10 03:52:22.420416 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:22.420388 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-8c28k"] Sep 10 03:52:22.458793 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:22.458758 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:22.509086 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:22.509044 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/9dd5377f-00ea-40d8-9673-b81aed6c6811-signing-key\") pod \"service-ca-75d99c484-8c28k\" (UID: \"9dd5377f-00ea-40d8-9673-b81aed6c6811\") " pod="openshift-service-ca/service-ca-75d99c484-8c28k" Sep 10 03:52:22.509220 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:22.509124 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/9dd5377f-00ea-40d8-9673-b81aed6c6811-signing-cabundle\") pod \"service-ca-75d99c484-8c28k\" (UID: \"9dd5377f-00ea-40d8-9673-b81aed6c6811\") " pod="openshift-service-ca/service-ca-75d99c484-8c28k" Sep 10 03:52:22.509275 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:22.509235 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h67km\" (UniqueName: \"kubernetes.io/projected/9dd5377f-00ea-40d8-9673-b81aed6c6811-kube-api-access-h67km\") pod \"service-ca-75d99c484-8c28k\" (UID: \"9dd5377f-00ea-40d8-9673-b81aed6c6811\") " pod="openshift-service-ca/service-ca-75d99c484-8c28k" Sep 10 03:52:22.514315 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:22.514274 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:22.609807 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:22.609772 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h67km\" (UniqueName: \"kubernetes.io/projected/9dd5377f-00ea-40d8-9673-b81aed6c6811-kube-api-access-h67km\") pod \"service-ca-75d99c484-8c28k\" (UID: \"9dd5377f-00ea-40d8-9673-b81aed6c6811\") " pod="openshift-service-ca/service-ca-75d99c484-8c28k" Sep 10 03:52:22.609998 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:22.609831 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/9dd5377f-00ea-40d8-9673-b81aed6c6811-signing-key\") pod \"service-ca-75d99c484-8c28k\" (UID: \"9dd5377f-00ea-40d8-9673-b81aed6c6811\") " pod="openshift-service-ca/service-ca-75d99c484-8c28k" Sep 10 03:52:22.609998 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:22.609983 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/9dd5377f-00ea-40d8-9673-b81aed6c6811-signing-cabundle\") pod \"service-ca-75d99c484-8c28k\" (UID: \"9dd5377f-00ea-40d8-9673-b81aed6c6811\") " pod="openshift-service-ca/service-ca-75d99c484-8c28k" Sep 10 03:52:22.610639 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:22.610620 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/9dd5377f-00ea-40d8-9673-b81aed6c6811-signing-cabundle\") pod \"service-ca-75d99c484-8c28k\" (UID: \"9dd5377f-00ea-40d8-9673-b81aed6c6811\") " pod="openshift-service-ca/service-ca-75d99c484-8c28k" Sep 10 03:52:22.612243 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:22.612223 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/9dd5377f-00ea-40d8-9673-b81aed6c6811-signing-key\") pod \"service-ca-75d99c484-8c28k\" (UID: \"9dd5377f-00ea-40d8-9673-b81aed6c6811\") " pod="openshift-service-ca/service-ca-75d99c484-8c28k" Sep 10 03:52:22.617748 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:22.617717 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h67km\" (UniqueName: \"kubernetes.io/projected/9dd5377f-00ea-40d8-9673-b81aed6c6811-kube-api-access-h67km\") pod \"service-ca-75d99c484-8c28k\" (UID: \"9dd5377f-00ea-40d8-9673-b81aed6c6811\") " pod="openshift-service-ca/service-ca-75d99c484-8c28k" Sep 10 03:52:22.724158 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:22.724068 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-8c28k" Sep 10 03:52:22.913178 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:22.913142 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/678609c3-b3fa-4beb-953d-d61067e32212-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-q5p2x\" (UID: \"678609c3-b3fa-4beb-953d-d61067e32212\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-q5p2x" Sep 10 03:52:22.913178 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:22.913181 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3b13de7-3c45-4da0-b80d-06508774147c-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-jlq9n\" (UID: \"b3b13de7-3c45-4da0-b80d-06508774147c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jlq9n" Sep 10 03:52:22.913410 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:22.913279 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 03:52:22.913410 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:22.913284 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 03:52:22.913410 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:22.913351 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3b13de7-3c45-4da0-b80d-06508774147c-samples-operator-tls podName:b3b13de7-3c45-4da0-b80d-06508774147c nodeName:}" failed. No retries permitted until 2026-09-10 03:52:30.913336553 +0000 UTC m=+277.789399683 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3b13de7-3c45-4da0-b80d-06508774147c-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-jlq9n" (UID: "b3b13de7-3c45-4da0-b80d-06508774147c") : secret "samples-operator-tls" not found Sep 10 03:52:22.913410 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:22.913364 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/678609c3-b3fa-4beb-953d-d61067e32212-cluster-monitoring-operator-tls podName:678609c3-b3fa-4beb-953d-d61067e32212 nodeName:}" failed. No retries permitted until 2026-09-10 03:52:30.913357842 +0000 UTC m=+277.789420971 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/678609c3-b3fa-4beb-953d-d61067e32212-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-q5p2x" (UID: "678609c3-b3fa-4beb-953d-d61067e32212") : secret "cluster-monitoring-operator-tls" not found Sep 10 03:52:23.007070 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:23.006979 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-8c28k"] Sep 10 03:52:23.204062 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:23.204021 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nmf2n_1ef729ce-06d8-4c1d-ac8c-7a89770df17f/node-ca/0.log" Sep 10 03:52:23.235000 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:52:23.234965 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dd5377f_00ea_40d8_9673_b81aed6c6811.slice/crio-ad48a943899c138fdb2405847ed6c413e5d1a5000705b727de36b07e8f1c49e4 WatchSource:0}: Error finding container ad48a943899c138fdb2405847ed6c413e5d1a5000705b727de36b07e8f1c49e4: Status 404 returned error can't find the container with id ad48a943899c138fdb2405847ed6c413e5d1a5000705b727de36b07e8f1c49e4 Sep 10 03:52:23.305089 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:23.305060 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:23.404942 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:23.404903 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-8c28k" event={"ID":"9dd5377f-00ea-40d8-9673-b81aed6c6811","Type":"ContainerStarted","Data":"ad48a943899c138fdb2405847ed6c413e5d1a5000705b727de36b07e8f1c49e4"} Sep 10 03:52:23.406776 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:23.406746 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-gz76m" event={"ID":"4f351318-0f05-4b9c-9266-73b5ca137fa8","Type":"ContainerStarted","Data":"32a31154f5be72fdb7ffd128a115a51199e807aac8afc9e3ad949aded1b9cf0f"} Sep 10 03:52:23.406933 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:23.406798 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-gz76m" event={"ID":"4f351318-0f05-4b9c-9266-73b5ca137fa8","Type":"ContainerStarted","Data":"0c72c9ca3020c18a6d70536f3e0223b036c11e0d067a1eb31dbb81d23ca59d08"} Sep 10 03:52:23.423406 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:23.423354 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-gz76m" podStartSLOduration=0.986770903 podStartE2EDuration="2.423339029s" totalImagesPullingTime="1.436568126s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:52:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:52:23.4223429 +0000 UTC m=+270.298406049" watchObservedRunningTime="2026-09-10 03:52:23.423339029 +0000 UTC m=+270.299402181" Sep 10 03:52:24.021907 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:24.021869 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/74b614da-e905-4ff7-9bfe-2d9bea24859b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-54dww\" (UID: \"74b614da-e905-4ff7-9bfe-2d9bea24859b\") " pod="openshift-insights/insights-runtime-extractor-54dww" Sep 10 03:52:24.022118 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:24.022013 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 03:52:24.022118 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:24.022064 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/74b614da-e905-4ff7-9bfe-2d9bea24859b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-54dww\" (UID: \"74b614da-e905-4ff7-9bfe-2d9bea24859b\") " pod="openshift-insights/insights-runtime-extractor-54dww" Sep 10 03:52:24.022118 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:24.022090 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/74b614da-e905-4ff7-9bfe-2d9bea24859b-kube-rbac-proxy-cm podName:74b614da-e905-4ff7-9bfe-2d9bea24859b nodeName:}" failed. No retries permitted until 2026-09-10 03:52:28.022071364 +0000 UTC m=+274.898134498 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/74b614da-e905-4ff7-9bfe-2d9bea24859b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-54dww" (UID: "74b614da-e905-4ff7-9bfe-2d9bea24859b") : configmap "kube-rbac-proxy" not found Sep 10 03:52:24.022266 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:24.022161 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 03:52:24.022266 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:24.022210 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74b614da-e905-4ff7-9bfe-2d9bea24859b-insights-runtime-extractor-tls podName:74b614da-e905-4ff7-9bfe-2d9bea24859b nodeName:}" failed. No retries permitted until 2026-09-10 03:52:28.022198983 +0000 UTC m=+274.898262113 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/74b614da-e905-4ff7-9bfe-2d9bea24859b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-54dww" (UID: "74b614da-e905-4ff7-9bfe-2d9bea24859b") : secret "insights-runtime-extractor-tls" not found Sep 10 03:52:24.202977 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:24.202946 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kx9jc_e6305575-8975-44bd-bd29-866c8457a56c/serve-healthcheck-canary/0.log" Sep 10 03:52:25.414441 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:25.414404 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-8c28k" event={"ID":"9dd5377f-00ea-40d8-9673-b81aed6c6811","Type":"ContainerStarted","Data":"0da97230fb300712af759ac90b8ba289ae0c7a5e187e11961391e9973657be63"} Sep 10 03:52:25.430476 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:25.430419 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-75d99c484-8c28k" podStartSLOduration=1.661281027 podStartE2EDuration="3.430400672s" totalImagesPullingTime="1.769119645s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:52:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:52:25.429092922 +0000 UTC m=+272.305156075" watchObservedRunningTime="2026-09-10 03:52:25.430400672 +0000 UTC m=+272.306463824" Sep 10 03:52:25.519582 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:25.519547 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-hf7wl" Sep 10 03:52:25.519683 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:25.519607 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-hf7wl" Sep 10 03:52:25.519962 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:25.519948 2562 scope.go:117] "RemoveContainer" containerID="b0a70a1556f90bdabaf180edf973a9895b85dc9b60378441d1da37245c2c1416" Sep 10 03:52:25.520112 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:25.520094 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-hf7wl_openshift-console-operator(50451d7c-07fb-4641-a36f-d142e72b5e81)\"" pod="openshift-console-operator/console-operator-85b454874b-hf7wl" podUID="50451d7c-07fb-4641-a36f-d142e72b5e81" Sep 10 03:52:26.128871 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:26.128831 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:26.258662 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:26.258614 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:26.316001 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:26.315952 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:26.382016 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:26.381924 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:26.504308 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:26.504252 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:26.584445 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:26.584408 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:26.685868 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:26.685838 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:26.827020 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:26.826981 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:28.031874 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:28.031837 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:28.057050 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:28.057017 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/74b614da-e905-4ff7-9bfe-2d9bea24859b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-54dww\" (UID: \"74b614da-e905-4ff7-9bfe-2d9bea24859b\") " pod="openshift-insights/insights-runtime-extractor-54dww" Sep 10 03:52:28.057159 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:28.057080 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/74b614da-e905-4ff7-9bfe-2d9bea24859b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-54dww\" (UID: \"74b614da-e905-4ff7-9bfe-2d9bea24859b\") " pod="openshift-insights/insights-runtime-extractor-54dww" Sep 10 03:52:28.057200 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:28.057166 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 03:52:28.057242 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:28.057224 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74b614da-e905-4ff7-9bfe-2d9bea24859b-insights-runtime-extractor-tls podName:74b614da-e905-4ff7-9bfe-2d9bea24859b nodeName:}" failed. No retries permitted until 2026-09-10 03:52:36.057209702 +0000 UTC m=+282.933272836 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/74b614da-e905-4ff7-9bfe-2d9bea24859b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-54dww" (UID: "74b614da-e905-4ff7-9bfe-2d9bea24859b") : secret "insights-runtime-extractor-tls" not found Sep 10 03:52:28.057315 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:28.057172 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 03:52:28.057363 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:28.057316 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/74b614da-e905-4ff7-9bfe-2d9bea24859b-kube-rbac-proxy-cm podName:74b614da-e905-4ff7-9bfe-2d9bea24859b nodeName:}" failed. No retries permitted until 2026-09-10 03:52:36.057283349 +0000 UTC m=+282.933346480 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/74b614da-e905-4ff7-9bfe-2d9bea24859b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-54dww" (UID: "74b614da-e905-4ff7-9bfe-2d9bea24859b") : configmap "kube-rbac-proxy" not found Sep 10 03:52:28.229212 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:28.229173 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:28.424028 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:28.423989 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:28.623877 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:28.623838 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:28.829197 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:28.829105 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:29.796152 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:29.796117 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:29.848165 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:29.848133 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:30.021912 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:30.021875 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:30.230972 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:30.230927 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:30.425842 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:30.425814 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:30.627797 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:30.627713 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:30.981811 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:30.981781 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/678609c3-b3fa-4beb-953d-d61067e32212-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-q5p2x\" (UID: \"678609c3-b3fa-4beb-953d-d61067e32212\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-q5p2x" Sep 10 03:52:30.982220 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:30.981817 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3b13de7-3c45-4da0-b80d-06508774147c-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-jlq9n\" (UID: \"b3b13de7-3c45-4da0-b80d-06508774147c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jlq9n" Sep 10 03:52:30.982220 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:30.981937 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 03:52:30.982220 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:30.982003 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/678609c3-b3fa-4beb-953d-d61067e32212-cluster-monitoring-operator-tls podName:678609c3-b3fa-4beb-953d-d61067e32212 nodeName:}" failed. No retries permitted until 2026-09-10 03:52:46.981988104 +0000 UTC m=+293.858051234 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/678609c3-b3fa-4beb-953d-d61067e32212-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-q5p2x" (UID: "678609c3-b3fa-4beb-953d-d61067e32212") : secret "cluster-monitoring-operator-tls" not found Sep 10 03:52:30.984308 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:30.984263 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3b13de7-3c45-4da0-b80d-06508774147c-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-jlq9n\" (UID: \"b3b13de7-3c45-4da0-b80d-06508774147c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jlq9n" Sep 10 03:52:31.019178 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:31.019148 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:31.113744 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:31.113690 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jlq9n" Sep 10 03:52:31.385058 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:31.384927 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jlq9n"] Sep 10 03:52:31.419402 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:31.419372 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:32.020747 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:32.020716 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:32.422105 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:32.422072 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:32.438496 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:32.438462 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jlq9n" event={"ID":"b3b13de7-3c45-4da0-b80d-06508774147c","Type":"ContainerStarted","Data":"843f74e9e6e5f0da5a49254042cd1bc362fc7f8fceaae438857ad6816b7c5c29"} Sep 10 03:52:32.621137 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:32.621097 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:32.680713 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:32.680586 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:33.023669 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:33.023572 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:33.498287 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:33.498218 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:33.702328 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:33.702235 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:34.024843 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:34.024738 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:34.113004 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:34.112967 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-79977f9844-2mjrb" podUID="aa9e7159-181a-4a1f-afe1-33cd08e9ba6a" Sep 10 03:52:34.113004 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:34.112968 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" podUID="4ba38a7e-088d-4fdf-840c-c85b7826f1a6" Sep 10 03:52:34.113157 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:34.112983 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-dkkkk" podUID="897fe719-a91a-4d75-8efb-4cb2fbade932" Sep 10 03:52:34.424738 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:34.424696 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:34.447564 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:34.447527 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jlq9n" event={"ID":"b3b13de7-3c45-4da0-b80d-06508774147c","Type":"ContainerStarted","Data":"2c2d35df3e0084cecaaad982f67ff8556c9b0cf1d3bfb3758aae507d6eff4297"} Sep 10 03:52:34.447723 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:34.447571 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dkkkk" Sep 10 03:52:34.447723 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:34.447608 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:52:34.447723 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:34.447625 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jlq9n" event={"ID":"b3b13de7-3c45-4da0-b80d-06508774147c","Type":"ContainerStarted","Data":"ffaf090f5e65c4a547dd5931ec5e83b1371729630e6836204040df9b312d0ce6"} Sep 10 03:52:34.447723 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:34.447571 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-2mjrb" Sep 10 03:52:34.463686 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:34.463635 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jlq9n" podStartSLOduration=17.474007422 podStartE2EDuration="19.463616774s" totalImagesPullingTime="1.989609352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:52:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:52:34.46200747 +0000 UTC m=+281.338070617" watchObservedRunningTime="2026-09-10 03:52:34.463616774 +0000 UTC m=+281.339679925" Sep 10 03:52:34.817743 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:34.817659 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:35.223419 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:35.223386 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:35.620880 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:35.620797 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:36.022644 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:36.022613 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:36.125162 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:36.125119 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/74b614da-e905-4ff7-9bfe-2d9bea24859b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-54dww\" (UID: \"74b614da-e905-4ff7-9bfe-2d9bea24859b\") " pod="openshift-insights/insights-runtime-extractor-54dww" Sep 10 03:52:36.125377 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:36.125277 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 03:52:36.125441 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:36.125376 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74b614da-e905-4ff7-9bfe-2d9bea24859b-insights-runtime-extractor-tls podName:74b614da-e905-4ff7-9bfe-2d9bea24859b nodeName:}" failed. No retries permitted until 2026-09-10 03:52:52.1253546 +0000 UTC m=+299.001417747 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/74b614da-e905-4ff7-9bfe-2d9bea24859b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-54dww" (UID: "74b614da-e905-4ff7-9bfe-2d9bea24859b") : secret "insights-runtime-extractor-tls" not found Sep 10 03:52:36.125441 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:36.125283 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/74b614da-e905-4ff7-9bfe-2d9bea24859b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-54dww\" (UID: \"74b614da-e905-4ff7-9bfe-2d9bea24859b\") " pod="openshift-insights/insights-runtime-extractor-54dww" Sep 10 03:52:36.125766 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:36.125745 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/74b614da-e905-4ff7-9bfe-2d9bea24859b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-54dww\" (UID: \"74b614da-e905-4ff7-9bfe-2d9bea24859b\") " pod="openshift-insights/insights-runtime-extractor-54dww" Sep 10 03:52:36.220160 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:36.220125 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:36.631579 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:36.631545 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:36.824865 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:36.824830 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:37.437912 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:37.437869 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-tls\") pod \"image-registry-554d6765f7-ss9hs\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:52:37.437912 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:37.437912 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/aa9e7159-181a-4a1f-afe1-33cd08e9ba6a-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-2mjrb\" (UID: \"aa9e7159-181a-4a1f-afe1-33cd08e9ba6a\") " pod="openshift-network-console/networking-console-plugin-79977f9844-2mjrb" Sep 10 03:52:37.440548 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:37.440512 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-tls\") pod \"image-registry-554d6765f7-ss9hs\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:52:37.440683 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:37.440514 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/aa9e7159-181a-4a1f-afe1-33cd08e9ba6a-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-2mjrb\" (UID: \"aa9e7159-181a-4a1f-afe1-33cd08e9ba6a\") " pod="openshift-network-console/networking-console-plugin-79977f9844-2mjrb" Sep 10 03:52:37.450787 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:37.450737 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-lfmcw\"" Sep 10 03:52:37.450787 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:37.450737 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-qn2w9\"" Sep 10 03:52:37.458432 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:37.458411 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:52:37.458578 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:37.458483 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-2mjrb" Sep 10 03:52:37.538920 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:37.538810 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/897fe719-a91a-4d75-8efb-4cb2fbade932-metrics-tls\") pod \"dns-default-dkkkk\" (UID: \"897fe719-a91a-4d75-8efb-4cb2fbade932\") " pod="openshift-dns/dns-default-dkkkk" Sep 10 03:52:37.541589 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:37.541560 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/897fe719-a91a-4d75-8efb-4cb2fbade932-metrics-tls\") pod \"dns-default-dkkkk\" (UID: \"897fe719-a91a-4d75-8efb-4cb2fbade932\") " pod="openshift-dns/dns-default-dkkkk" Sep 10 03:52:37.737798 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:37.737718 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-2mjrb"] Sep 10 03:52:37.740798 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:52:37.740761 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa9e7159_181a_4a1f_afe1_33cd08e9ba6a.slice/crio-9c212dd5aadf83ecf0b53ade87c507574caeb37761ed0fc9a971b77a731d7725 WatchSource:0}: Error finding container 9c212dd5aadf83ecf0b53ade87c507574caeb37761ed0fc9a971b77a731d7725: Status 404 returned error can't find the container with id 9c212dd5aadf83ecf0b53ade87c507574caeb37761ed0fc9a971b77a731d7725 Sep 10 03:52:37.741195 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:37.741176 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-554d6765f7-ss9hs"] Sep 10 03:52:37.744492 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:52:37.744463 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ba38a7e_088d_4fdf_840c_c85b7826f1a6.slice/crio-37081ba1cc6a888841292096b7576890158421cb4703eb1c33ca8a13228ba939 WatchSource:0}: Error finding container 37081ba1cc6a888841292096b7576890158421cb4703eb1c33ca8a13228ba939: Status 404 returned error can't find the container with id 37081ba1cc6a888841292096b7576890158421cb4703eb1c33ca8a13228ba939 Sep 10 03:52:37.750351 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:37.750325 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bt2fn\"" Sep 10 03:52:37.759052 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:37.759001 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dkkkk" Sep 10 03:52:38.033130 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:38.033052 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dkkkk"] Sep 10 03:52:38.036128 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:52:38.036101 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod897fe719_a91a_4d75_8efb_4cb2fbade932.slice/crio-e69734a69fddff9f27aaedaa25d0c6f1e4dfeaf30e386b93d4f59b6d94cd24eb WatchSource:0}: Error finding container e69734a69fddff9f27aaedaa25d0c6f1e4dfeaf30e386b93d4f59b6d94cd24eb: Status 404 returned error can't find the container with id e69734a69fddff9f27aaedaa25d0c6f1e4dfeaf30e386b93d4f59b6d94cd24eb Sep 10 03:52:38.461472 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:38.461422 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dkkkk" event={"ID":"897fe719-a91a-4d75-8efb-4cb2fbade932","Type":"ContainerStarted","Data":"e69734a69fddff9f27aaedaa25d0c6f1e4dfeaf30e386b93d4f59b6d94cd24eb"} Sep 10 03:52:38.463008 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:38.462976 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-2mjrb" event={"ID":"aa9e7159-181a-4a1f-afe1-33cd08e9ba6a","Type":"ContainerStarted","Data":"9c212dd5aadf83ecf0b53ade87c507574caeb37761ed0fc9a971b77a731d7725"} Sep 10 03:52:38.465244 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:38.465212 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" event={"ID":"4ba38a7e-088d-4fdf-840c-c85b7826f1a6","Type":"ContainerStarted","Data":"c90abd1fb5515d4c48843e37e24005c83a38113cf074d5e28c1510bc4d05b46e"} Sep 10 03:52:38.465392 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:38.465267 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" event={"ID":"4ba38a7e-088d-4fdf-840c-c85b7826f1a6","Type":"ContainerStarted","Data":"37081ba1cc6a888841292096b7576890158421cb4703eb1c33ca8a13228ba939"} Sep 10 03:52:38.465392 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:38.465385 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:52:38.486586 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:38.486529 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" podStartSLOduration=282.486509881 podStartE2EDuration="4m42.486509881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:47:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:52:38.484957366 +0000 UTC m=+285.361020519" watchObservedRunningTime="2026-09-10 03:52:38.486509881 +0000 UTC m=+285.362573032" Sep 10 03:52:38.599125 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:38.599088 2562 scope.go:117] "RemoveContainer" containerID="b0a70a1556f90bdabaf180edf973a9895b85dc9b60378441d1da37245c2c1416" Sep 10 03:52:39.470238 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:39.470209 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hf7wl_50451d7c-07fb-4641-a36f-d142e72b5e81/console-operator/2.log" Sep 10 03:52:39.470756 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:39.470734 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hf7wl_50451d7c-07fb-4641-a36f-d142e72b5e81/console-operator/1.log" Sep 10 03:52:39.470807 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:39.470781 2562 generic.go:356] "Generic (PLEG): container finished" podID="50451d7c-07fb-4641-a36f-d142e72b5e81" containerID="af698bf524a47edca2fc092e5ed3f7f5dbebf122ae1be27592ab90373bd5ddaa" exitCode=255 Sep 10 03:52:39.470902 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:39.470868 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-hf7wl" event={"ID":"50451d7c-07fb-4641-a36f-d142e72b5e81","Type":"ContainerDied","Data":"af698bf524a47edca2fc092e5ed3f7f5dbebf122ae1be27592ab90373bd5ddaa"} Sep 10 03:52:39.470994 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:39.470974 2562 scope.go:117] "RemoveContainer" containerID="b0a70a1556f90bdabaf180edf973a9895b85dc9b60378441d1da37245c2c1416" Sep 10 03:52:39.471519 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:39.471403 2562 scope.go:117] "RemoveContainer" containerID="af698bf524a47edca2fc092e5ed3f7f5dbebf122ae1be27592ab90373bd5ddaa" Sep 10 03:52:39.471648 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:39.471622 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-hf7wl_openshift-console-operator(50451d7c-07fb-4641-a36f-d142e72b5e81)\"" pod="openshift-console-operator/console-operator-85b454874b-hf7wl" podUID="50451d7c-07fb-4641-a36f-d142e72b5e81" Sep 10 03:52:39.472664 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:39.472637 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-2mjrb" event={"ID":"aa9e7159-181a-4a1f-afe1-33cd08e9ba6a","Type":"ContainerStarted","Data":"f7f484852a6e65193032f474c020b405753b80f657a98c3084579509e3eb1bad"} Sep 10 03:52:39.502189 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:39.502123 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-2mjrb" podStartSLOduration=256.231745139 podStartE2EDuration="4m17.502099515s" totalImagesPullingTime="1.270354376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:48:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:52:39.501187546 +0000 UTC m=+286.377250693" watchObservedRunningTime="2026-09-10 03:52:39.502099515 +0000 UTC m=+286.378162670" Sep 10 03:52:40.477960 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:40.477932 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hf7wl_50451d7c-07fb-4641-a36f-d142e72b5e81/console-operator/2.log" Sep 10 03:52:40.479798 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:40.479765 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dkkkk" event={"ID":"897fe719-a91a-4d75-8efb-4cb2fbade932","Type":"ContainerStarted","Data":"87f46e346b713fdb0176ede94dc864c9b64f407582ff19e115f7496783f5f642"} Sep 10 03:52:40.479950 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:40.479818 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dkkkk" event={"ID":"897fe719-a91a-4d75-8efb-4cb2fbade932","Type":"ContainerStarted","Data":"5656f8da5435958676989cf4e81641c4309c9090a14be3a497e5afb81f60a264"} Sep 10 03:52:40.496940 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:40.496880 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-dkkkk" podStartSLOduration=251.854153343 podStartE2EDuration="4m13.496863204s" totalImagesPullingTime="1.642709861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:48:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:52:40.495404435 +0000 UTC m=+287.371467587" watchObservedRunningTime="2026-09-10 03:52:40.496863204 +0000 UTC m=+287.372926356" Sep 10 03:52:41.228860 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:41.228828 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:41.326612 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:41.326579 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:41.483811 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:41.483725 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dkkkk" Sep 10 03:52:41.524990 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:41.524956 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-554d6765f7-ss9hs"] Sep 10 03:52:41.565513 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:41.565473 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6f7cc79777-ccvgm"] Sep 10 03:52:41.568981 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:41.568959 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f7cc79777-ccvgm" Sep 10 03:52:41.573643 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:41.573620 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/17499bec-4013-4b5d-9d0c-008ee3db1541-image-registry-private-configuration\") pod \"image-registry-6f7cc79777-ccvgm\" (UID: \"17499bec-4013-4b5d-9d0c-008ee3db1541\") " pod="openshift-image-registry/image-registry-6f7cc79777-ccvgm" Sep 10 03:52:41.573742 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:41.573647 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/17499bec-4013-4b5d-9d0c-008ee3db1541-registry-tls\") pod \"image-registry-6f7cc79777-ccvgm\" (UID: \"17499bec-4013-4b5d-9d0c-008ee3db1541\") " pod="openshift-image-registry/image-registry-6f7cc79777-ccvgm" Sep 10 03:52:41.573742 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:41.573721 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6twf\" (UniqueName: \"kubernetes.io/projected/17499bec-4013-4b5d-9d0c-008ee3db1541-kube-api-access-p6twf\") pod \"image-registry-6f7cc79777-ccvgm\" (UID: \"17499bec-4013-4b5d-9d0c-008ee3db1541\") " pod="openshift-image-registry/image-registry-6f7cc79777-ccvgm" Sep 10 03:52:41.573828 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:41.573816 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/17499bec-4013-4b5d-9d0c-008ee3db1541-registry-certificates\") pod \"image-registry-6f7cc79777-ccvgm\" (UID: \"17499bec-4013-4b5d-9d0c-008ee3db1541\") " pod="openshift-image-registry/image-registry-6f7cc79777-ccvgm" Sep 10 03:52:41.573865 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:41.573833 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/17499bec-4013-4b5d-9d0c-008ee3db1541-trusted-ca\") pod \"image-registry-6f7cc79777-ccvgm\" (UID: \"17499bec-4013-4b5d-9d0c-008ee3db1541\") " pod="openshift-image-registry/image-registry-6f7cc79777-ccvgm" Sep 10 03:52:41.573906 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:41.573873 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/17499bec-4013-4b5d-9d0c-008ee3db1541-installation-pull-secrets\") pod \"image-registry-6f7cc79777-ccvgm\" (UID: \"17499bec-4013-4b5d-9d0c-008ee3db1541\") " pod="openshift-image-registry/image-registry-6f7cc79777-ccvgm" Sep 10 03:52:41.574017 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:41.573996 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/17499bec-4013-4b5d-9d0c-008ee3db1541-bound-sa-token\") pod \"image-registry-6f7cc79777-ccvgm\" (UID: \"17499bec-4013-4b5d-9d0c-008ee3db1541\") " pod="openshift-image-registry/image-registry-6f7cc79777-ccvgm" Sep 10 03:52:41.574091 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:41.574058 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/17499bec-4013-4b5d-9d0c-008ee3db1541-ca-trust-extracted\") pod \"image-registry-6f7cc79777-ccvgm\" (UID: \"17499bec-4013-4b5d-9d0c-008ee3db1541\") " pod="openshift-image-registry/image-registry-6f7cc79777-ccvgm" Sep 10 03:52:41.590598 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:41.590572 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6f7cc79777-ccvgm"] Sep 10 03:52:41.633574 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:41.633538 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:41.675017 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:41.674981 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/17499bec-4013-4b5d-9d0c-008ee3db1541-registry-certificates\") pod \"image-registry-6f7cc79777-ccvgm\" (UID: \"17499bec-4013-4b5d-9d0c-008ee3db1541\") " pod="openshift-image-registry/image-registry-6f7cc79777-ccvgm" Sep 10 03:52:41.675017 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:41.675015 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/17499bec-4013-4b5d-9d0c-008ee3db1541-trusted-ca\") pod \"image-registry-6f7cc79777-ccvgm\" (UID: \"17499bec-4013-4b5d-9d0c-008ee3db1541\") " pod="openshift-image-registry/image-registry-6f7cc79777-ccvgm" Sep 10 03:52:41.675234 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:41.675067 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/17499bec-4013-4b5d-9d0c-008ee3db1541-installation-pull-secrets\") pod \"image-registry-6f7cc79777-ccvgm\" (UID: \"17499bec-4013-4b5d-9d0c-008ee3db1541\") " pod="openshift-image-registry/image-registry-6f7cc79777-ccvgm" Sep 10 03:52:41.675234 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:41.675099 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/17499bec-4013-4b5d-9d0c-008ee3db1541-bound-sa-token\") pod \"image-registry-6f7cc79777-ccvgm\" (UID: \"17499bec-4013-4b5d-9d0c-008ee3db1541\") " pod="openshift-image-registry/image-registry-6f7cc79777-ccvgm" Sep 10 03:52:41.675330 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:41.675240 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/17499bec-4013-4b5d-9d0c-008ee3db1541-ca-trust-extracted\") pod \"image-registry-6f7cc79777-ccvgm\" (UID: \"17499bec-4013-4b5d-9d0c-008ee3db1541\") " pod="openshift-image-registry/image-registry-6f7cc79777-ccvgm" Sep 10 03:52:41.675387 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:41.675316 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/17499bec-4013-4b5d-9d0c-008ee3db1541-image-registry-private-configuration\") pod \"image-registry-6f7cc79777-ccvgm\" (UID: \"17499bec-4013-4b5d-9d0c-008ee3db1541\") " pod="openshift-image-registry/image-registry-6f7cc79777-ccvgm" Sep 10 03:52:41.675437 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:41.675386 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/17499bec-4013-4b5d-9d0c-008ee3db1541-registry-tls\") pod \"image-registry-6f7cc79777-ccvgm\" (UID: \"17499bec-4013-4b5d-9d0c-008ee3db1541\") " pod="openshift-image-registry/image-registry-6f7cc79777-ccvgm" Sep 10 03:52:41.675493 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:41.675436 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6twf\" (UniqueName: \"kubernetes.io/projected/17499bec-4013-4b5d-9d0c-008ee3db1541-kube-api-access-p6twf\") pod \"image-registry-6f7cc79777-ccvgm\" (UID: \"17499bec-4013-4b5d-9d0c-008ee3db1541\") " pod="openshift-image-registry/image-registry-6f7cc79777-ccvgm" Sep 10 03:52:41.676335 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:41.676240 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/17499bec-4013-4b5d-9d0c-008ee3db1541-ca-trust-extracted\") pod \"image-registry-6f7cc79777-ccvgm\" (UID: \"17499bec-4013-4b5d-9d0c-008ee3db1541\") " pod="openshift-image-registry/image-registry-6f7cc79777-ccvgm" Sep 10 03:52:41.676478 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:41.676449 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/17499bec-4013-4b5d-9d0c-008ee3db1541-trusted-ca\") pod \"image-registry-6f7cc79777-ccvgm\" (UID: \"17499bec-4013-4b5d-9d0c-008ee3db1541\") " pod="openshift-image-registry/image-registry-6f7cc79777-ccvgm" Sep 10 03:52:41.676589 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:41.676566 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/17499bec-4013-4b5d-9d0c-008ee3db1541-registry-certificates\") pod \"image-registry-6f7cc79777-ccvgm\" (UID: \"17499bec-4013-4b5d-9d0c-008ee3db1541\") " pod="openshift-image-registry/image-registry-6f7cc79777-ccvgm" Sep 10 03:52:41.679180 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:41.679145 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/17499bec-4013-4b5d-9d0c-008ee3db1541-installation-pull-secrets\") pod \"image-registry-6f7cc79777-ccvgm\" (UID: \"17499bec-4013-4b5d-9d0c-008ee3db1541\") " pod="openshift-image-registry/image-registry-6f7cc79777-ccvgm" Sep 10 03:52:41.679354 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:41.679178 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/17499bec-4013-4b5d-9d0c-008ee3db1541-registry-tls\") pod \"image-registry-6f7cc79777-ccvgm\" (UID: \"17499bec-4013-4b5d-9d0c-008ee3db1541\") " pod="openshift-image-registry/image-registry-6f7cc79777-ccvgm" Sep 10 03:52:41.682386 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:41.682357 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/17499bec-4013-4b5d-9d0c-008ee3db1541-image-registry-private-configuration\") pod \"image-registry-6f7cc79777-ccvgm\" (UID: \"17499bec-4013-4b5d-9d0c-008ee3db1541\") " pod="openshift-image-registry/image-registry-6f7cc79777-ccvgm" Sep 10 03:52:41.687181 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:41.687156 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6twf\" (UniqueName: \"kubernetes.io/projected/17499bec-4013-4b5d-9d0c-008ee3db1541-kube-api-access-p6twf\") pod \"image-registry-6f7cc79777-ccvgm\" (UID: \"17499bec-4013-4b5d-9d0c-008ee3db1541\") " pod="openshift-image-registry/image-registry-6f7cc79777-ccvgm" Sep 10 03:52:41.687280 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:41.687244 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/17499bec-4013-4b5d-9d0c-008ee3db1541-bound-sa-token\") pod \"image-registry-6f7cc79777-ccvgm\" (UID: \"17499bec-4013-4b5d-9d0c-008ee3db1541\") " pod="openshift-image-registry/image-registry-6f7cc79777-ccvgm" Sep 10 03:52:41.736872 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:41.736728 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:41.840865 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:41.840825 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:41.878257 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:41.878223 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f7cc79777-ccvgm" Sep 10 03:52:41.934144 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:41.934109 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:42.151960 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:42.151925 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6f7cc79777-ccvgm"] Sep 10 03:52:42.154714 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:52:42.154685 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17499bec_4013_4b5d_9d0c_008ee3db1541.slice/crio-e9bfb39bcd2596ff50060c7d9a994bc30d22230f1d2fc9756d38a885fc6398b7 WatchSource:0}: Error finding container e9bfb39bcd2596ff50060c7d9a994bc30d22230f1d2fc9756d38a885fc6398b7: Status 404 returned error can't find the container with id e9bfb39bcd2596ff50060c7d9a994bc30d22230f1d2fc9756d38a885fc6398b7 Sep 10 03:52:42.489550 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:42.489463 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f7cc79777-ccvgm" event={"ID":"17499bec-4013-4b5d-9d0c-008ee3db1541","Type":"ContainerStarted","Data":"d429487ea0c63c0f6153cc69a1f9bfa3ba4f303a9282c928bb08e46bcd9b322f"} Sep 10 03:52:42.489550 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:42.489512 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f7cc79777-ccvgm" event={"ID":"17499bec-4013-4b5d-9d0c-008ee3db1541","Type":"ContainerStarted","Data":"e9bfb39bcd2596ff50060c7d9a994bc30d22230f1d2fc9756d38a885fc6398b7"} Sep 10 03:52:42.489941 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:42.489625 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6f7cc79777-ccvgm" Sep 10 03:52:42.509135 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:42.509081 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6f7cc79777-ccvgm" podStartSLOduration=1.509063321 podStartE2EDuration="1.509063321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:52:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:52:42.508222848 +0000 UTC m=+289.384286001" watchObservedRunningTime="2026-09-10 03:52:42.509063321 +0000 UTC m=+289.385126474" Sep 10 03:52:45.518966 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:45.518914 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-hf7wl" Sep 10 03:52:45.519401 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:45.518996 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-hf7wl" Sep 10 03:52:45.519401 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:45.519341 2562 scope.go:117] "RemoveContainer" containerID="af698bf524a47edca2fc092e5ed3f7f5dbebf122ae1be27592ab90373bd5ddaa" Sep 10 03:52:45.519506 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:45.519486 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-hf7wl_openshift-console-operator(50451d7c-07fb-4641-a36f-d142e72b5e81)\"" pod="openshift-console-operator/console-operator-85b454874b-hf7wl" podUID="50451d7c-07fb-4641-a36f-d142e72b5e81" Sep 10 03:52:46.599995 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:46.599953 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-2p9b8" podUID="158bac16-c713-47cd-91d7-9fc36d96f570" Sep 10 03:52:47.011954 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:47.011906 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/678609c3-b3fa-4beb-953d-d61067e32212-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-q5p2x\" (UID: \"678609c3-b3fa-4beb-953d-d61067e32212\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-q5p2x" Sep 10 03:52:47.014386 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:47.014353 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/678609c3-b3fa-4beb-953d-d61067e32212-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-q5p2x\" (UID: \"678609c3-b3fa-4beb-953d-d61067e32212\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-q5p2x" Sep 10 03:52:47.306340 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:47.306228 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-q5p2x" Sep 10 03:52:47.577308 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:47.577210 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-q5p2x"] Sep 10 03:52:47.580682 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:52:47.580651 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod678609c3_b3fa_4beb_953d_d61067e32212.slice/crio-245c165be78921d920f6509bc20a3c884b9deca2e50482d6b6764da643e6c43b WatchSource:0}: Error finding container 245c165be78921d920f6509bc20a3c884b9deca2e50482d6b6764da643e6c43b: Status 404 returned error can't find the container with id 245c165be78921d920f6509bc20a3c884b9deca2e50482d6b6764da643e6c43b Sep 10 03:52:48.512058 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:48.512019 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-q5p2x" event={"ID":"678609c3-b3fa-4beb-953d-d61067e32212","Type":"ContainerStarted","Data":"245c165be78921d920f6509bc20a3c884b9deca2e50482d6b6764da643e6c43b"} Sep 10 03:52:49.517631 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:49.517588 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-q5p2x" event={"ID":"678609c3-b3fa-4beb-953d-d61067e32212","Type":"ContainerStarted","Data":"c78b5bc52f2ae347a277ec34e776935bf87dd965f28c30cb3ee056b93e553c86"} Sep 10 03:52:49.532162 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:49.532102 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-q5p2x" podStartSLOduration=32.691838709 podStartE2EDuration="34.532079735s" totalImagesPullingTime="1.840241026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:52:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:52:49.532080973 +0000 UTC m=+296.408144126" watchObservedRunningTime="2026-09-10 03:52:49.532079735 +0000 UTC m=+296.408142882" Sep 10 03:52:49.761000 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:49.760902 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:49.861343 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:49.861282 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:49.932470 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:49.932433 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:50.206479 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:50.206446 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:51.491764 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:51.491734 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dkkkk" Sep 10 03:52:52.154444 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:52.154407 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/74b614da-e905-4ff7-9bfe-2d9bea24859b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-54dww\" (UID: \"74b614da-e905-4ff7-9bfe-2d9bea24859b\") " pod="openshift-insights/insights-runtime-extractor-54dww" Sep 10 03:52:52.156798 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:52.156751 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/74b614da-e905-4ff7-9bfe-2d9bea24859b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-54dww\" (UID: \"74b614da-e905-4ff7-9bfe-2d9bea24859b\") " pod="openshift-insights/insights-runtime-extractor-54dww" Sep 10 03:52:52.355633 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:52.355561 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-54dww" Sep 10 03:52:52.629989 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:52.629952 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-54dww"] Sep 10 03:52:52.632978 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:52:52.632951 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b614da_e905_4ff7_9bfe_2d9bea24859b.slice/crio-be4f28169a270f539ba5be3e6725504a4ac151633f5231eb17d47426160c1d01 WatchSource:0}: Error finding container be4f28169a270f539ba5be3e6725504a4ac151633f5231eb17d47426160c1d01: Status 404 returned error can't find the container with id be4f28169a270f539ba5be3e6725504a4ac151633f5231eb17d47426160c1d01 Sep 10 03:52:52.941626 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:52.941542 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:53.017055 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:53.017021 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:53.331203 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:53.331115 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:53.514671 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:53.514646 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hf7wl_50451d7c-07fb-4641-a36f-d142e72b5e81/console-operator/2.log" Sep 10 03:52:53.515286 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:53.515255 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hf7wl_50451d7c-07fb-4641-a36f-d142e72b5e81/console-operator/2.log" Sep 10 03:52:53.520111 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:53.520083 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fl7hj_8459a996-8d64-4c23-9e92-1a5bcca253b5/ovn-acl-logging/0.log" Sep 10 03:52:53.520720 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:53.520694 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fl7hj_8459a996-8d64-4c23-9e92-1a5bcca253b5/ovn-acl-logging/0.log" Sep 10 03:52:53.526587 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:53.526560 2562 kubelet.go:1648] "Image garbage collection succeeded" Sep 10 03:52:53.534578 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:53.534546 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-54dww" event={"ID":"74b614da-e905-4ff7-9bfe-2d9bea24859b","Type":"ContainerStarted","Data":"c5c91087c56e5fb8d802ed82431201808840c60a3c4429fff092ae60c84078fc"} Sep 10 03:52:53.534694 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:53.534603 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-54dww" event={"ID":"74b614da-e905-4ff7-9bfe-2d9bea24859b","Type":"ContainerStarted","Data":"be4f28169a270f539ba5be3e6725504a4ac151633f5231eb17d47426160c1d01"} Sep 10 03:52:54.540436 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:54.540398 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-54dww" event={"ID":"74b614da-e905-4ff7-9bfe-2d9bea24859b","Type":"ContainerStarted","Data":"b22e682b7265e4d8aa06186ba807d059f4da9e7b8b715940bab72e4b9fda5b9a"} Sep 10 03:52:55.546455 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:55.546418 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-54dww" event={"ID":"74b614da-e905-4ff7-9bfe-2d9bea24859b","Type":"ContainerStarted","Data":"d591b56875d9eef5bbceaa0b8c9db443c9d1c3951cca22ff71352690fee7fbee"} Sep 10 03:52:55.563495 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:55.563438 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-54dww" podStartSLOduration=33.377877182 podStartE2EDuration="35.563421219s" totalImagesPullingTime="2.185544037s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 03:52:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:52:55.562924906 +0000 UTC m=+302.438988055" watchObservedRunningTime="2026-09-10 03:52:55.563421219 +0000 UTC m=+302.439484362" Sep 10 03:52:55.614880 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:55.614844 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:56.598866 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:56.598832 2562 scope.go:117] "RemoveContainer" containerID="af698bf524a47edca2fc092e5ed3f7f5dbebf122ae1be27592ab90373bd5ddaa" Sep 10 03:52:56.599235 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:56.598997 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-hf7wl_openshift-console-operator(50451d7c-07fb-4641-a36f-d142e72b5e81)\"" pod="openshift-console-operator/console-operator-85b454874b-hf7wl" podUID="50451d7c-07fb-4641-a36f-d142e72b5e81" Sep 10 03:52:56.896302 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:56.896269 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:58.075624 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.075592 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:58.312380 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.312336 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:58.351002 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.350883 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-sbhps"] Sep 10 03:52:58.353731 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.353712 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sbhps" Sep 10 03:52:58.356478 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.356449 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 10 03:52:58.356892 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.356872 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 10 03:52:58.357349 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.357330 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-s6wbd\"" Sep 10 03:52:58.357605 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.357590 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 10 03:52:58.357727 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.357612 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 10 03:52:58.395933 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.395893 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:58.405045 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.405005 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd-sys\") pod \"node-exporter-sbhps\" (UID: \"3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd\") " pod="openshift-monitoring/node-exporter-sbhps" Sep 10 03:52:58.405236 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.405066 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd-node-exporter-textfile\") pod \"node-exporter-sbhps\" (UID: \"3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd\") " pod="openshift-monitoring/node-exporter-sbhps" Sep 10 03:52:58.405236 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.405202 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd-node-exporter-tls\") pod \"node-exporter-sbhps\" (UID: \"3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd\") " pod="openshift-monitoring/node-exporter-sbhps" Sep 10 03:52:58.405373 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.405239 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd-metrics-client-ca\") pod \"node-exporter-sbhps\" (UID: \"3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd\") " pod="openshift-monitoring/node-exporter-sbhps" Sep 10 03:52:58.405373 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.405308 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mnjv\" (UniqueName: \"kubernetes.io/projected/3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd-kube-api-access-2mnjv\") pod \"node-exporter-sbhps\" (UID: \"3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd\") " pod="openshift-monitoring/node-exporter-sbhps" Sep 10 03:52:58.405465 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.405368 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sbhps\" (UID: \"3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd\") " pod="openshift-monitoring/node-exporter-sbhps" Sep 10 03:52:58.405465 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.405404 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd-node-exporter-accelerators-collector-config\") pod \"node-exporter-sbhps\" (UID: \"3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd\") " pod="openshift-monitoring/node-exporter-sbhps" Sep 10 03:52:58.405465 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.405432 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd-node-exporter-wtmp\") pod \"node-exporter-sbhps\" (UID: \"3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd\") " pod="openshift-monitoring/node-exporter-sbhps" Sep 10 03:52:58.405465 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.405458 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd-root\") pod \"node-exporter-sbhps\" (UID: \"3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd\") " pod="openshift-monitoring/node-exporter-sbhps" Sep 10 03:52:58.467364 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.467329 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:58.506627 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.506581 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sbhps\" (UID: \"3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd\") " pod="openshift-monitoring/node-exporter-sbhps" Sep 10 03:52:58.506627 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.506632 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd-node-exporter-accelerators-collector-config\") pod \"node-exporter-sbhps\" (UID: \"3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd\") " pod="openshift-monitoring/node-exporter-sbhps" Sep 10 03:52:58.506923 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.506658 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd-node-exporter-wtmp\") pod \"node-exporter-sbhps\" (UID: \"3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd\") " pod="openshift-monitoring/node-exporter-sbhps" Sep 10 03:52:58.506923 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.506681 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd-root\") pod \"node-exporter-sbhps\" (UID: \"3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd\") " pod="openshift-monitoring/node-exporter-sbhps" Sep 10 03:52:58.506923 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.506729 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd-sys\") pod \"node-exporter-sbhps\" (UID: \"3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd\") " pod="openshift-monitoring/node-exporter-sbhps" Sep 10 03:52:58.506923 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.506749 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd-node-exporter-textfile\") pod \"node-exporter-sbhps\" (UID: \"3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd\") " pod="openshift-monitoring/node-exporter-sbhps" Sep 10 03:52:58.506923 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.506823 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd-node-exporter-wtmp\") pod \"node-exporter-sbhps\" (UID: \"3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd\") " pod="openshift-monitoring/node-exporter-sbhps" Sep 10 03:52:58.506923 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.506857 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd-sys\") pod \"node-exporter-sbhps\" (UID: \"3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd\") " pod="openshift-monitoring/node-exporter-sbhps" Sep 10 03:52:58.506923 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.506824 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd-root\") pod \"node-exporter-sbhps\" (UID: \"3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd\") " pod="openshift-monitoring/node-exporter-sbhps" Sep 10 03:52:58.506923 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.506882 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd-node-exporter-tls\") pod \"node-exporter-sbhps\" (UID: \"3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd\") " pod="openshift-monitoring/node-exporter-sbhps" Sep 10 03:52:58.506923 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.506903 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd-metrics-client-ca\") pod \"node-exporter-sbhps\" (UID: \"3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd\") " pod="openshift-monitoring/node-exporter-sbhps" Sep 10 03:52:58.506923 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.506929 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mnjv\" (UniqueName: \"kubernetes.io/projected/3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd-kube-api-access-2mnjv\") pod \"node-exporter-sbhps\" (UID: \"3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd\") " pod="openshift-monitoring/node-exporter-sbhps" Sep 10 03:52:58.507441 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:58.507024 2562 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 10 03:52:58.507441 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:52:58.507097 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd-node-exporter-tls podName:3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd nodeName:}" failed. No retries permitted until 2026-09-10 03:52:59.007076385 +0000 UTC m=+305.883139528 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd-node-exporter-tls") pod "node-exporter-sbhps" (UID: "3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd") : secret "node-exporter-tls" not found Sep 10 03:52:58.507441 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.507345 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd-node-exporter-accelerators-collector-config\") pod \"node-exporter-sbhps\" (UID: \"3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd\") " pod="openshift-monitoring/node-exporter-sbhps" Sep 10 03:52:58.507739 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.507551 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd-node-exporter-textfile\") pod \"node-exporter-sbhps\" (UID: \"3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd\") " pod="openshift-monitoring/node-exporter-sbhps" Sep 10 03:52:58.507739 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.507551 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd-metrics-client-ca\") pod \"node-exporter-sbhps\" (UID: \"3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd\") " pod="openshift-monitoring/node-exporter-sbhps" Sep 10 03:52:58.509647 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.509626 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sbhps\" (UID: \"3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd\") " pod="openshift-monitoring/node-exporter-sbhps" Sep 10 03:52:58.517209 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.517183 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mnjv\" (UniqueName: \"kubernetes.io/projected/3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd-kube-api-access-2mnjv\") pod \"node-exporter-sbhps\" (UID: \"3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd\") " pod="openshift-monitoring/node-exporter-sbhps" Sep 10 03:52:58.564918 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.564878 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:58.679731 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.679698 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:58.780447 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.780412 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:58.881330 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.881280 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:58.980798 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:58.980707 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:59.010093 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.010060 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd-node-exporter-tls\") pod \"node-exporter-sbhps\" (UID: \"3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd\") " pod="openshift-monitoring/node-exporter-sbhps" Sep 10 03:52:59.012489 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.012460 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd-node-exporter-tls\") pod \"node-exporter-sbhps\" (UID: \"3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd\") " pod="openshift-monitoring/node-exporter-sbhps" Sep 10 03:52:59.265522 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.265425 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sbhps" Sep 10 03:52:59.278954 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:52:59.278920 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3af1ec86_b6f3_4d16_bb0e_16cb77a80dfd.slice/crio-8a95af1feaba3a6fc4b170602725243618717a15cd09c6e7128a76ee805c3e53 WatchSource:0}: Error finding container 8a95af1feaba3a6fc4b170602725243618717a15cd09c6e7128a76ee805c3e53: Status 404 returned error can't find the container with id 8a95af1feaba3a6fc4b170602725243618717a15cd09c6e7128a76ee805c3e53 Sep 10 03:52:59.281629 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.281605 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 03:52:59.305404 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.305361 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:59.378729 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.378693 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 03:52:59.379273 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.379248 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:59.382396 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.382378 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.384579 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.384549 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 10 03:52:59.385229 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.384889 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 10 03:52:59.385229 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.384907 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 10 03:52:59.385229 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.384929 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 10 03:52:59.385229 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.384964 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 10 03:52:59.385229 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.385101 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 10 03:52:59.385653 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.385280 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 10 03:52:59.385653 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.385323 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 10 03:52:59.385653 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.385351 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 10 03:52:59.385653 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.385352 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-5sld2\"" Sep 10 03:52:59.396059 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.396031 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 03:52:59.412798 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.412754 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4283ea87-d273-4def-a3c7-c0a520715be0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.412798 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.412799 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-web-config\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.413001 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.412823 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.413001 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.412842 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.413001 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.412906 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4283ea87-d273-4def-a3c7-c0a520715be0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.413001 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.412971 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-config-volume\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.413205 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.413014 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4283ea87-d273-4def-a3c7-c0a520715be0-config-out\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.413205 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.413040 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4283ea87-d273-4def-a3c7-c0a520715be0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.413205 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.413072 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.413205 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.413105 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.413205 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.413154 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.413205 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.413178 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4283ea87-d273-4def-a3c7-c0a520715be0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.413205 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.413201 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xqd4\" (UniqueName: \"kubernetes.io/projected/4283ea87-d273-4def-a3c7-c0a520715be0-kube-api-access-5xqd4\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.513630 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.513599 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4283ea87-d273-4def-a3c7-c0a520715be0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.513630 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.513628 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-web-config\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.513851 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.513647 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.513851 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.513718 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:52:59.513851 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.513808 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.513851 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.513842 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4283ea87-d273-4def-a3c7-c0a520715be0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.514154 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.513877 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-config-volume\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.514154 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.513908 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4283ea87-d273-4def-a3c7-c0a520715be0-config-out\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.514154 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.514023 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4283ea87-d273-4def-a3c7-c0a520715be0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.514154 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.514062 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.514154 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.514097 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.514154 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.514134 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.514465 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.514160 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4283ea87-d273-4def-a3c7-c0a520715be0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.514465 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.514184 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xqd4\" (UniqueName: \"kubernetes.io/projected/4283ea87-d273-4def-a3c7-c0a520715be0-kube-api-access-5xqd4\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.514942 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.514757 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4283ea87-d273-4def-a3c7-c0a520715be0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.515134 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.515109 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4283ea87-d273-4def-a3c7-c0a520715be0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.515935 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.515873 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4283ea87-d273-4def-a3c7-c0a520715be0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.516806 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.516776 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4283ea87-d273-4def-a3c7-c0a520715be0-config-out\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.517143 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.517098 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4283ea87-d273-4def-a3c7-c0a520715be0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.517233 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.517154 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-web-config\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.517898 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.517872 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.517997 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.517972 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.518268 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.518238 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.518532 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.518509 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.519231 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.519197 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-config-volume\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.519482 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.519465 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.522807 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.522786 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xqd4\" (UniqueName: \"kubernetes.io/projected/4283ea87-d273-4def-a3c7-c0a520715be0-kube-api-access-5xqd4\") pod \"alertmanager-main-0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.562117 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.562081 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sbhps" event={"ID":"3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd","Type":"ContainerStarted","Data":"8a95af1feaba3a6fc4b170602725243618717a15cd09c6e7128a76ee805c3e53"} Sep 10 03:52:59.693028 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.692984 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:52:59.979372 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:52:59.979343 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 03:52:59.981395 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:52:59.981364 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4283ea87_d273_4def_a3c7_c0a520715be0.slice/crio-ccfb457f587791ea2ec52743bb4af4476c1b7e8e6811d3eab00385b143c76f37 WatchSource:0}: Error finding container ccfb457f587791ea2ec52743bb4af4476c1b7e8e6811d3eab00385b143c76f37: Status 404 returned error can't find the container with id ccfb457f587791ea2ec52743bb4af4476c1b7e8e6811d3eab00385b143c76f37 Sep 10 03:53:00.328008 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.327934 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-65894fbfc9-v8z5g"] Sep 10 03:53:00.331353 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.331332 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-65894fbfc9-v8z5g" Sep 10 03:53:00.333710 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.333675 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Sep 10 03:53:00.333865 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.333843 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Sep 10 03:53:00.333865 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.333854 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Sep 10 03:53:00.333984 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.333859 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Sep 10 03:53:00.333984 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.333886 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-q4cs8\"" Sep 10 03:53:00.333984 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.333918 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-1dh9vuagjm4lf\"" Sep 10 03:53:00.333984 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.333923 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Sep 10 03:53:00.343934 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.343909 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-65894fbfc9-v8z5g"] Sep 10 03:53:00.422684 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.422639 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4d5fd4c5-9178-4ae2-994a-445fa67e6174-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-65894fbfc9-v8z5g\" (UID: \"4d5fd4c5-9178-4ae2-994a-445fa67e6174\") " pod="openshift-monitoring/thanos-querier-65894fbfc9-v8z5g" Sep 10 03:53:00.422684 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.422691 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4d5fd4c5-9178-4ae2-994a-445fa67e6174-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-65894fbfc9-v8z5g\" (UID: \"4d5fd4c5-9178-4ae2-994a-445fa67e6174\") " pod="openshift-monitoring/thanos-querier-65894fbfc9-v8z5g" Sep 10 03:53:00.422949 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.422724 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d5fd4c5-9178-4ae2-994a-445fa67e6174-metrics-client-ca\") pod \"thanos-querier-65894fbfc9-v8z5g\" (UID: \"4d5fd4c5-9178-4ae2-994a-445fa67e6174\") " pod="openshift-monitoring/thanos-querier-65894fbfc9-v8z5g" Sep 10 03:53:00.422949 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.422752 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4d5fd4c5-9178-4ae2-994a-445fa67e6174-secret-grpc-tls\") pod \"thanos-querier-65894fbfc9-v8z5g\" (UID: \"4d5fd4c5-9178-4ae2-994a-445fa67e6174\") " pod="openshift-monitoring/thanos-querier-65894fbfc9-v8z5g" Sep 10 03:53:00.422949 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.422806 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4d5fd4c5-9178-4ae2-994a-445fa67e6174-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-65894fbfc9-v8z5g\" (UID: \"4d5fd4c5-9178-4ae2-994a-445fa67e6174\") " pod="openshift-monitoring/thanos-querier-65894fbfc9-v8z5g" Sep 10 03:53:00.422949 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.422899 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrk9z\" (UniqueName: \"kubernetes.io/projected/4d5fd4c5-9178-4ae2-994a-445fa67e6174-kube-api-access-jrk9z\") pod \"thanos-querier-65894fbfc9-v8z5g\" (UID: \"4d5fd4c5-9178-4ae2-994a-445fa67e6174\") " pod="openshift-monitoring/thanos-querier-65894fbfc9-v8z5g" Sep 10 03:53:00.423145 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.422966 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4d5fd4c5-9178-4ae2-994a-445fa67e6174-secret-thanos-querier-tls\") pod \"thanos-querier-65894fbfc9-v8z5g\" (UID: \"4d5fd4c5-9178-4ae2-994a-445fa67e6174\") " pod="openshift-monitoring/thanos-querier-65894fbfc9-v8z5g" Sep 10 03:53:00.423145 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.423009 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4d5fd4c5-9178-4ae2-994a-445fa67e6174-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-65894fbfc9-v8z5g\" (UID: \"4d5fd4c5-9178-4ae2-994a-445fa67e6174\") " pod="openshift-monitoring/thanos-querier-65894fbfc9-v8z5g" Sep 10 03:53:00.523907 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.523874 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d5fd4c5-9178-4ae2-994a-445fa67e6174-metrics-client-ca\") pod \"thanos-querier-65894fbfc9-v8z5g\" (UID: \"4d5fd4c5-9178-4ae2-994a-445fa67e6174\") " pod="openshift-monitoring/thanos-querier-65894fbfc9-v8z5g" Sep 10 03:53:00.524062 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.523916 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4d5fd4c5-9178-4ae2-994a-445fa67e6174-secret-grpc-tls\") pod \"thanos-querier-65894fbfc9-v8z5g\" (UID: \"4d5fd4c5-9178-4ae2-994a-445fa67e6174\") " pod="openshift-monitoring/thanos-querier-65894fbfc9-v8z5g" Sep 10 03:53:00.524062 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.523987 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4d5fd4c5-9178-4ae2-994a-445fa67e6174-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-65894fbfc9-v8z5g\" (UID: \"4d5fd4c5-9178-4ae2-994a-445fa67e6174\") " pod="openshift-monitoring/thanos-querier-65894fbfc9-v8z5g" Sep 10 03:53:00.524062 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.524048 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrk9z\" (UniqueName: \"kubernetes.io/projected/4d5fd4c5-9178-4ae2-994a-445fa67e6174-kube-api-access-jrk9z\") pod \"thanos-querier-65894fbfc9-v8z5g\" (UID: \"4d5fd4c5-9178-4ae2-994a-445fa67e6174\") " pod="openshift-monitoring/thanos-querier-65894fbfc9-v8z5g" Sep 10 03:53:00.524218 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.524093 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4d5fd4c5-9178-4ae2-994a-445fa67e6174-secret-thanos-querier-tls\") pod \"thanos-querier-65894fbfc9-v8z5g\" (UID: \"4d5fd4c5-9178-4ae2-994a-445fa67e6174\") " pod="openshift-monitoring/thanos-querier-65894fbfc9-v8z5g" Sep 10 03:53:00.524218 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.524135 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4d5fd4c5-9178-4ae2-994a-445fa67e6174-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-65894fbfc9-v8z5g\" (UID: \"4d5fd4c5-9178-4ae2-994a-445fa67e6174\") " pod="openshift-monitoring/thanos-querier-65894fbfc9-v8z5g" Sep 10 03:53:00.524347 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.524177 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4d5fd4c5-9178-4ae2-994a-445fa67e6174-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-65894fbfc9-v8z5g\" (UID: \"4d5fd4c5-9178-4ae2-994a-445fa67e6174\") " pod="openshift-monitoring/thanos-querier-65894fbfc9-v8z5g" Sep 10 03:53:00.524347 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.524268 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4d5fd4c5-9178-4ae2-994a-445fa67e6174-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-65894fbfc9-v8z5g\" (UID: \"4d5fd4c5-9178-4ae2-994a-445fa67e6174\") " pod="openshift-monitoring/thanos-querier-65894fbfc9-v8z5g" Sep 10 03:53:00.525066 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.525037 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d5fd4c5-9178-4ae2-994a-445fa67e6174-metrics-client-ca\") pod \"thanos-querier-65894fbfc9-v8z5g\" (UID: \"4d5fd4c5-9178-4ae2-994a-445fa67e6174\") " pod="openshift-monitoring/thanos-querier-65894fbfc9-v8z5g" Sep 10 03:53:00.527367 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.527256 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4d5fd4c5-9178-4ae2-994a-445fa67e6174-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-65894fbfc9-v8z5g\" (UID: \"4d5fd4c5-9178-4ae2-994a-445fa67e6174\") " pod="openshift-monitoring/thanos-querier-65894fbfc9-v8z5g" Sep 10 03:53:00.527367 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.527276 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4d5fd4c5-9178-4ae2-994a-445fa67e6174-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-65894fbfc9-v8z5g\" (UID: \"4d5fd4c5-9178-4ae2-994a-445fa67e6174\") " pod="openshift-monitoring/thanos-querier-65894fbfc9-v8z5g" Sep 10 03:53:00.527552 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.527486 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4d5fd4c5-9178-4ae2-994a-445fa67e6174-secret-thanos-querier-tls\") pod \"thanos-querier-65894fbfc9-v8z5g\" (UID: \"4d5fd4c5-9178-4ae2-994a-445fa67e6174\") " pod="openshift-monitoring/thanos-querier-65894fbfc9-v8z5g" Sep 10 03:53:00.527701 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.527682 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4d5fd4c5-9178-4ae2-994a-445fa67e6174-secret-grpc-tls\") pod \"thanos-querier-65894fbfc9-v8z5g\" (UID: \"4d5fd4c5-9178-4ae2-994a-445fa67e6174\") " pod="openshift-monitoring/thanos-querier-65894fbfc9-v8z5g" Sep 10 03:53:00.527778 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.527712 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4d5fd4c5-9178-4ae2-994a-445fa67e6174-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-65894fbfc9-v8z5g\" (UID: \"4d5fd4c5-9178-4ae2-994a-445fa67e6174\") " pod="openshift-monitoring/thanos-querier-65894fbfc9-v8z5g" Sep 10 03:53:00.527778 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.527730 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4d5fd4c5-9178-4ae2-994a-445fa67e6174-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-65894fbfc9-v8z5g\" (UID: \"4d5fd4c5-9178-4ae2-994a-445fa67e6174\") " pod="openshift-monitoring/thanos-querier-65894fbfc9-v8z5g" Sep 10 03:53:00.531816 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.531790 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrk9z\" (UniqueName: \"kubernetes.io/projected/4d5fd4c5-9178-4ae2-994a-445fa67e6174-kube-api-access-jrk9z\") pod \"thanos-querier-65894fbfc9-v8z5g\" (UID: \"4d5fd4c5-9178-4ae2-994a-445fa67e6174\") " pod="openshift-monitoring/thanos-querier-65894fbfc9-v8z5g" Sep 10 03:53:00.569102 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.569068 2562 generic.go:356] "Generic (PLEG): container finished" podID="3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd" containerID="3b2eb31735fba7f7952c120f496cce4414216e72eb8eeab03c1357fa38251333" exitCode=0 Sep 10 03:53:00.569313 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.569160 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sbhps" event={"ID":"3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd","Type":"ContainerDied","Data":"3b2eb31735fba7f7952c120f496cce4414216e72eb8eeab03c1357fa38251333"} Sep 10 03:53:00.571027 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.571001 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4283ea87-d273-4def-a3c7-c0a520715be0","Type":"ContainerStarted","Data":"ccfb457f587791ea2ec52743bb4af4476c1b7e8e6811d3eab00385b143c76f37"} Sep 10 03:53:00.598569 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.598479 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2p9b8" Sep 10 03:53:00.640869 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.640834 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-65894fbfc9-v8z5g" Sep 10 03:53:00.656865 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.656572 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:00.758476 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.758377 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:00.942047 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:00.942012 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-65894fbfc9-v8z5g"] Sep 10 03:53:01.072210 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:53:01.072116 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d5fd4c5_9178_4ae2_994a_445fa67e6174.slice/crio-ffbf111a6524e1b6670851618a51a8256278871454e054a6349b1c407f92c786 WatchSource:0}: Error finding container ffbf111a6524e1b6670851618a51a8256278871454e054a6349b1c407f92c786: Status 404 returned error can't find the container with id ffbf111a6524e1b6670851618a51a8256278871454e054a6349b1c407f92c786 Sep 10 03:53:01.280877 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:01.280846 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:01.381891 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:01.381852 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:01.533530 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:01.533449 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:53:01.577116 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:01.577077 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65894fbfc9-v8z5g" event={"ID":"4d5fd4c5-9178-4ae2-994a-445fa67e6174","Type":"ContainerStarted","Data":"ffbf111a6524e1b6670851618a51a8256278871454e054a6349b1c407f92c786"} Sep 10 03:53:01.579729 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:01.579690 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sbhps" event={"ID":"3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd","Type":"ContainerStarted","Data":"24bb91b0efd1ac8ceb6601ce644e98a33c58f73e8d065c1a2f6abe581008c170"} Sep 10 03:53:01.579918 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:01.579755 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sbhps" event={"ID":"3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd","Type":"ContainerStarted","Data":"10183133887e3aa8dbd27b80d805c890dba3d092a0c9959407a1915a00980d90"} Sep 10 03:53:01.581492 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:01.581463 2562 generic.go:356] "Generic (PLEG): container finished" podID="4283ea87-d273-4def-a3c7-c0a520715be0" containerID="f9da984ae568936fee7cea52cf8f1e00edc25334774925e65ea8f4c0719fc077" exitCode=0 Sep 10 03:53:01.581613 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:01.581503 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4283ea87-d273-4def-a3c7-c0a520715be0","Type":"ContainerDied","Data":"f9da984ae568936fee7cea52cf8f1e00edc25334774925e65ea8f4c0719fc077"} Sep 10 03:53:01.597905 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:01.597840 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-sbhps" podStartSLOduration=2.472466131 podStartE2EDuration="3.597816207s" totalImagesPullingTime="1.058103225s" totalInitContainerRuntime="67.246851ms" isStatefulPod=true podCreationTimestamp="2026-09-10 03:52:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:53:01.596202652 +0000 UTC m=+308.472265805" watchObservedRunningTime="2026-09-10 03:53:01.597816207 +0000 UTC m=+308.473879360" Sep 10 03:53:01.814708 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:01.814614 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:01.914055 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:01.914018 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:03.102810 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:03.102775 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:03.203321 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:03.203274 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:03.497812 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:03.497791 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6f7cc79777-ccvgm" Sep 10 03:53:03.605269 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:03.605234 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4283ea87-d273-4def-a3c7-c0a520715be0","Type":"ContainerStarted","Data":"d2954efa93096a780e7fb9ecd626991a96e16db96d0afecd40964f517b5a1853"} Sep 10 03:53:03.605426 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:03.605337 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65894fbfc9-v8z5g" event={"ID":"4d5fd4c5-9178-4ae2-994a-445fa67e6174","Type":"ContainerStarted","Data":"b03aa23ec6510d697e167a95ecde6b2083d431e9a9e7f4aca6f236bd6a54b169"} Sep 10 03:53:03.639988 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:03.639950 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:03.753776 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:03.753751 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:03.856751 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:03.856716 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:03.954099 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:03.954065 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:04.550001 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.549976 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:04.555156 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.555133 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 03:53:04.560421 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.560404 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.562779 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.562761 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-lnc3oa1mqbvh\"" Sep 10 03:53:04.562878 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.562762 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 10 03:53:04.563098 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.563008 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 10 03:53:04.563098 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.563081 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 10 03:53:04.563385 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.563370 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 10 03:53:04.563466 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.563449 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 10 03:53:04.563466 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.563453 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 10 03:53:04.563631 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.563613 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 10 03:53:04.563670 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.563655 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 10 03:53:04.563892 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.563877 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 10 03:53:04.563972 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.563893 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 10 03:53:04.563972 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.563916 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 10 03:53:04.564114 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.564086 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 10 03:53:04.564175 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.564136 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 10 03:53:04.564224 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.564175 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 10 03:53:04.564624 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.564608 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-p26td\"" Sep 10 03:53:04.566645 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.566624 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 10 03:53:04.583777 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.583701 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 03:53:04.612365 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.612332 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4283ea87-d273-4def-a3c7-c0a520715be0","Type":"ContainerStarted","Data":"a10a681dad7099d1b357e501d3f2f1141ba626e264525743f58b432fd394b80e"} Sep 10 03:53:04.612461 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.612419 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4283ea87-d273-4def-a3c7-c0a520715be0","Type":"ContainerStarted","Data":"76cbd3f933bfaa3c7e89b6ee2f5dc4f907352dede5256fe258bf0aa1ee0fd1c7"} Sep 10 03:53:04.612507 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.612465 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4283ea87-d273-4def-a3c7-c0a520715be0","Type":"ContainerStarted","Data":"cc4308d7932b88c2c96f855144295e9e0788e27e328172c3b82db46794465870"} Sep 10 03:53:04.612538 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.612504 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4283ea87-d273-4def-a3c7-c0a520715be0","Type":"ContainerStarted","Data":"ece18681196ceab502eb5a239f1fb645cf5c8eac408277eafc1681e8319a795c"} Sep 10 03:53:04.615730 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.615703 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65894fbfc9-v8z5g" event={"ID":"4d5fd4c5-9178-4ae2-994a-445fa67e6174","Type":"ContainerStarted","Data":"ab84476e8bd22be95371322bdc570ad82f2c6c470a4f91b2ca2528bb0cd7c5ca"} Sep 10 03:53:04.615830 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.615778 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65894fbfc9-v8z5g" event={"ID":"4d5fd4c5-9178-4ae2-994a-445fa67e6174","Type":"ContainerStarted","Data":"3d59ea10b66fdbee993a94bd5d9bad88049980a159c802efae8a60dcf8ada04a"} Sep 10 03:53:04.664036 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.663996 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.664232 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.664046 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.664232 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.664113 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.664232 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.664130 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t446c\" (UniqueName: \"kubernetes.io/projected/cfe42449-c4ae-4b63-be11-9f75095ed0a9-kube-api-access-t446c\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.664232 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.664146 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.664232 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.664169 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.664232 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.664189 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.664586 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.664254 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.664586 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.664284 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.664586 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.664345 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.664586 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.664364 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.664586 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.664385 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cfe42449-c4ae-4b63-be11-9f75095ed0a9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.664586 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.664409 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cfe42449-c4ae-4b63-be11-9f75095ed0a9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.664586 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.664439 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.664586 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.664461 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.664586 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.664505 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-web-config\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.664586 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.664540 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.664964 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.664618 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cfe42449-c4ae-4b63-be11-9f75095ed0a9-config-out\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.664964 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.664657 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-config\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.664964 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.664683 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.765819 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.765783 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.765980 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.765827 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.765980 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.765859 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.765980 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.765894 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.765980 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.765918 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.766184 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.766026 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.766184 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.766098 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.766319 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.766187 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cfe42449-c4ae-4b63-be11-9f75095ed0a9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.766319 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.766238 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cfe42449-c4ae-4b63-be11-9f75095ed0a9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.766421 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.766364 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.766421 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.766401 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.766515 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.766433 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-web-config\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.766515 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.766464 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.766515 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.766505 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cfe42449-c4ae-4b63-be11-9f75095ed0a9-config-out\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.766659 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.766533 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-config\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.766659 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.766556 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.766659 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.766593 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.766659 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.766617 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.766845 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.766671 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.766845 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.766695 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t446c\" (UniqueName: \"kubernetes.io/projected/cfe42449-c4ae-4b63-be11-9f75095ed0a9-kube-api-access-t446c\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.768226 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.767281 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.768226 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.767414 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.768226 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.767627 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cfe42449-c4ae-4b63-be11-9f75095ed0a9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.768226 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.767899 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.768226 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.768186 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.768904 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.768875 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.769172 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.769147 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.769934 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.769858 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.769934 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.769883 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.770713 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.770683 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.770834 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.770797 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-config\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.771420 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.771376 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.771504 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.771457 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.771688 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.771666 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cfe42449-c4ae-4b63-be11-9f75095ed0a9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.771776 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.771688 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.772191 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.772173 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.772322 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.772282 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cfe42449-c4ae-4b63-be11-9f75095ed0a9-config-out\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.772693 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.772676 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.773422 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.773406 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-web-config\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.775485 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.775466 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t446c\" (UniqueName: \"kubernetes.io/projected/cfe42449-c4ae-4b63-be11-9f75095ed0a9-kube-api-access-t446c\") pod \"prometheus-k8s-0\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:04.890762 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:04.890720 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:05.174868 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:05.174840 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 03:53:05.176751 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:53:05.176723 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfe42449_c4ae_4b63_be11_9f75095ed0a9.slice/crio-21b9a6daaa845d473b89b2633800b85cb4a1d0437d57d485a9162457def9ced5 WatchSource:0}: Error finding container 21b9a6daaa845d473b89b2633800b85cb4a1d0437d57d485a9162457def9ced5: Status 404 returned error can't find the container with id 21b9a6daaa845d473b89b2633800b85cb4a1d0437d57d485a9162457def9ced5 Sep 10 03:53:05.623147 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:05.623064 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4283ea87-d273-4def-a3c7-c0a520715be0","Type":"ContainerStarted","Data":"ed41d0ae8241eb3ade35726d4be15688f11b7875b74539680de5209076daf0cd"} Sep 10 03:53:05.626483 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:05.626451 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65894fbfc9-v8z5g" event={"ID":"4d5fd4c5-9178-4ae2-994a-445fa67e6174","Type":"ContainerStarted","Data":"3865977e15eda366f722468238dcb2c967cd1833c1c6e20bc0e6e4371eb71e7f"} Sep 10 03:53:05.626622 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:05.626523 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65894fbfc9-v8z5g" event={"ID":"4d5fd4c5-9178-4ae2-994a-445fa67e6174","Type":"ContainerStarted","Data":"ae6ebebcb36dc0969c9f06bfc9d2749d295a54e1720d51d3b11b83a2ba516b5b"} Sep 10 03:53:05.626622 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:05.626561 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65894fbfc9-v8z5g" event={"ID":"4d5fd4c5-9178-4ae2-994a-445fa67e6174","Type":"ContainerStarted","Data":"f6c85e572d462907e9ac1c1292493a3a66ea9f3d211934eb5e7803a624b355df"} Sep 10 03:53:05.626622 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:05.626602 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-65894fbfc9-v8z5g" Sep 10 03:53:05.628048 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:05.628020 2562 generic.go:356] "Generic (PLEG): container finished" podID="cfe42449-c4ae-4b63-be11-9f75095ed0a9" containerID="4577130ab15a0eb7bc6795d63610134a63fc6d62972e2cb8dcefec05ba3a1814" exitCode=0 Sep 10 03:53:05.628266 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:05.628064 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfe42449-c4ae-4b63-be11-9f75095ed0a9","Type":"ContainerDied","Data":"4577130ab15a0eb7bc6795d63610134a63fc6d62972e2cb8dcefec05ba3a1814"} Sep 10 03:53:05.628266 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:05.628107 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfe42449-c4ae-4b63-be11-9f75095ed0a9","Type":"ContainerStarted","Data":"21b9a6daaa845d473b89b2633800b85cb4a1d0437d57d485a9162457def9ced5"} Sep 10 03:53:05.651845 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:05.651797 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.88153609 podStartE2EDuration="6.651781508s" totalImagesPullingTime="3.751504942s" totalInitContainerRuntime="18.740476ms" isStatefulPod=false podCreationTimestamp="2026-09-10 03:52:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:53:05.650645149 +0000 UTC m=+312.526708300" watchObservedRunningTime="2026-09-10 03:53:05.651781508 +0000 UTC m=+312.527844660" Sep 10 03:53:05.672572 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:05.672513 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-65894fbfc9-v8z5g" podStartSLOduration=2.437919548 podStartE2EDuration="5.672494807s" totalImagesPullingTime="3.234575259s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:53:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:53:05.671161995 +0000 UTC m=+312.547225167" watchObservedRunningTime="2026-09-10 03:53:05.672494807 +0000 UTC m=+312.548557959" Sep 10 03:53:05.893185 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:05.893155 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:05.994084 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:05.994047 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:06.544631 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:06.544580 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" podUID="4ba38a7e-088d-4fdf-840c-c85b7826f1a6" containerName="registry" containerID="cri-o://c90abd1fb5515d4c48843e37e24005c83a38113cf074d5e28c1510bc4d05b46e" gracePeriod=30 Sep 10 03:53:06.806651 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:06.806573 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:53:06.889069 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:06.889030 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-installation-pull-secrets\") pod \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " Sep 10 03:53:06.889069 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:06.889077 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-trusted-ca\") pod \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " Sep 10 03:53:06.889331 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:06.889226 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sfg5p\" (UniqueName: \"kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-kube-api-access-sfg5p\") pod \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " Sep 10 03:53:06.889331 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:06.889253 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-bound-sa-token\") pod \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " Sep 10 03:53:06.889446 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:06.889342 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-ca-trust-extracted\") pod \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " Sep 10 03:53:06.889446 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:06.889373 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-certificates\") pod \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " Sep 10 03:53:06.889536 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:06.889473 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-image-registry-private-configuration\") pod \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " Sep 10 03:53:06.889536 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:06.889504 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-tls\") pod \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\" (UID: \"4ba38a7e-088d-4fdf-840c-c85b7826f1a6\") " Sep 10 03:53:06.889632 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:06.889527 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "4ba38a7e-088d-4fdf-840c-c85b7826f1a6" (UID: "4ba38a7e-088d-4fdf-840c-c85b7826f1a6"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 03:53:06.890408 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:06.889771 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-trusted-ca\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:53:06.890408 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:06.890370 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "4ba38a7e-088d-4fdf-840c-c85b7826f1a6" (UID: "4ba38a7e-088d-4fdf-840c-c85b7826f1a6"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 03:53:06.892230 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:06.892197 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "4ba38a7e-088d-4fdf-840c-c85b7826f1a6" (UID: "4ba38a7e-088d-4fdf-840c-c85b7826f1a6"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 03:53:06.892482 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:06.892376 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-kube-api-access-sfg5p" (OuterVolumeSpecName: "kube-api-access-sfg5p") pod "4ba38a7e-088d-4fdf-840c-c85b7826f1a6" (UID: "4ba38a7e-088d-4fdf-840c-c85b7826f1a6"). InnerVolumeSpecName "kube-api-access-sfg5p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 03:53:06.892482 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:06.892431 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4ba38a7e-088d-4fdf-840c-c85b7826f1a6" (UID: "4ba38a7e-088d-4fdf-840c-c85b7826f1a6"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 03:53:06.892989 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:06.892946 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "4ba38a7e-088d-4fdf-840c-c85b7826f1a6" (UID: "4ba38a7e-088d-4fdf-840c-c85b7826f1a6"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 03:53:06.893375 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:06.893336 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "4ba38a7e-088d-4fdf-840c-c85b7826f1a6" (UID: "4ba38a7e-088d-4fdf-840c-c85b7826f1a6"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 03:53:06.901280 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:06.901247 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "4ba38a7e-088d-4fdf-840c-c85b7826f1a6" (UID: "4ba38a7e-088d-4fdf-840c-c85b7826f1a6"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 03:53:06.991107 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:06.991064 2562 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-image-registry-private-configuration\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:53:06.991107 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:06.991096 2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-tls\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:53:06.991107 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:06.991110 2562 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-installation-pull-secrets\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:53:06.991500 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:06.991126 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfg5p\" (UniqueName: \"kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-kube-api-access-sfg5p\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:53:06.991500 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:06.991139 2562 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-bound-sa-token\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:53:06.991500 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:06.991148 2562 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-ca-trust-extracted\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:53:06.991500 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:06.991159 2562 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4ba38a7e-088d-4fdf-840c-c85b7826f1a6-registry-certificates\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:53:07.641566 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:07.641533 2562 generic.go:356] "Generic (PLEG): container finished" podID="4ba38a7e-088d-4fdf-840c-c85b7826f1a6" containerID="c90abd1fb5515d4c48843e37e24005c83a38113cf074d5e28c1510bc4d05b46e" exitCode=0 Sep 10 03:53:07.641752 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:07.641593 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" Sep 10 03:53:07.641752 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:07.641616 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" event={"ID":"4ba38a7e-088d-4fdf-840c-c85b7826f1a6","Type":"ContainerDied","Data":"c90abd1fb5515d4c48843e37e24005c83a38113cf074d5e28c1510bc4d05b46e"} Sep 10 03:53:07.641752 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:07.641680 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-554d6765f7-ss9hs" event={"ID":"4ba38a7e-088d-4fdf-840c-c85b7826f1a6","Type":"ContainerDied","Data":"37081ba1cc6a888841292096b7576890158421cb4703eb1c33ca8a13228ba939"} Sep 10 03:53:07.641752 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:07.641708 2562 scope.go:117] "RemoveContainer" containerID="c90abd1fb5515d4c48843e37e24005c83a38113cf074d5e28c1510bc4d05b46e" Sep 10 03:53:07.659513 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:07.659473 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-554d6765f7-ss9hs"] Sep 10 03:53:07.664714 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:07.664686 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-554d6765f7-ss9hs"] Sep 10 03:53:08.718188 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:08.717798 2562 scope.go:117] "RemoveContainer" containerID="c90abd1fb5515d4c48843e37e24005c83a38113cf074d5e28c1510bc4d05b46e" Sep 10 03:53:08.718188 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:53:08.718150 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c90abd1fb5515d4c48843e37e24005c83a38113cf074d5e28c1510bc4d05b46e\": container with ID starting with c90abd1fb5515d4c48843e37e24005c83a38113cf074d5e28c1510bc4d05b46e not found: ID does not exist" containerID="c90abd1fb5515d4c48843e37e24005c83a38113cf074d5e28c1510bc4d05b46e" Sep 10 03:53:08.718546 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:08.718199 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c90abd1fb5515d4c48843e37e24005c83a38113cf074d5e28c1510bc4d05b46e"} err="failed to get container status \"c90abd1fb5515d4c48843e37e24005c83a38113cf074d5e28c1510bc4d05b46e\": rpc error: code = NotFound desc = could not find container \"c90abd1fb5515d4c48843e37e24005c83a38113cf074d5e28c1510bc4d05b46e\": container with ID starting with c90abd1fb5515d4c48843e37e24005c83a38113cf074d5e28c1510bc4d05b46e not found: ID does not exist" Sep 10 03:53:09.599354 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:09.599285 2562 scope.go:117] "RemoveContainer" containerID="af698bf524a47edca2fc092e5ed3f7f5dbebf122ae1be27592ab90373bd5ddaa" Sep 10 03:53:09.604586 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:09.604562 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ba38a7e-088d-4fdf-840c-c85b7826f1a6" path="/var/lib/kubelet/pods/4ba38a7e-088d-4fdf-840c-c85b7826f1a6/volumes" Sep 10 03:53:09.662234 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:09.662194 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfe42449-c4ae-4b63-be11-9f75095ed0a9","Type":"ContainerStarted","Data":"b71778daae530b59ea7d814ad711c5ac7fe4cc44ce3ef9c5b7d82acfd7497577"} Sep 10 03:53:09.662418 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:09.662333 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfe42449-c4ae-4b63-be11-9f75095ed0a9","Type":"ContainerStarted","Data":"2961ec8c8dbbd6bb6227265d5fa1cef4e745facbd5e7bf664f48835d18947761"} Sep 10 03:53:09.662418 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:09.662382 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfe42449-c4ae-4b63-be11-9f75095ed0a9","Type":"ContainerStarted","Data":"0e7b980e97aa652270deb6f7744f9f4a01aede748e0ab96c199e82cf2c500340"} Sep 10 03:53:09.662535 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:09.662421 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfe42449-c4ae-4b63-be11-9f75095ed0a9","Type":"ContainerStarted","Data":"577b606f959b5114d614e5da30f1888c339ee14eed0faa21a8843effdd4bdf71"} Sep 10 03:53:09.662535 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:09.662461 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfe42449-c4ae-4b63-be11-9f75095ed0a9","Type":"ContainerStarted","Data":"202f84e17a4a55ac27d773c3fdf57add67a187b0c0deb43526a70936d16fed22"} Sep 10 03:53:09.662535 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:09.662492 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfe42449-c4ae-4b63-be11-9f75095ed0a9","Type":"ContainerStarted","Data":"428b753d61571c4dc502dd71c4f11f23a758bf525c5ef1ffa75d8c400a41b43f"} Sep 10 03:53:09.664148 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:09.664127 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hf7wl_50451d7c-07fb-4641-a36f-d142e72b5e81/console-operator/2.log" Sep 10 03:53:09.664271 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:09.664180 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-hf7wl" event={"ID":"50451d7c-07fb-4641-a36f-d142e72b5e81","Type":"ContainerStarted","Data":"740f1823d82bee057c33e2649da14504cf8cfdad835a0f828e970b882667edd9"} Sep 10 03:53:09.664533 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:09.664508 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-hf7wl" Sep 10 03:53:09.666178 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:09.666153 2562 patch_prober.go:28] interesting pod/console-operator-85b454874b-hf7wl container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.13:8443/readyz\": dial tcp 10.132.0.13:8443: connect: connection refused" start-of-body= Sep 10 03:53:09.666246 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:09.666205 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-85b454874b-hf7wl" podUID="50451d7c-07fb-4641-a36f-d142e72b5e81" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.13:8443/readyz\": dial tcp 10.132.0.13:8443: connect: connection refused" Sep 10 03:53:09.693173 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:09.693029 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.530125069 podStartE2EDuration="5.693008398s" totalImagesPullingTime="3.142437118s" totalInitContainerRuntime="20.446211ms" isStatefulPod=false podCreationTimestamp="2026-09-10 03:53:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:53:09.692153369 +0000 UTC m=+316.568216522" watchObservedRunningTime="2026-09-10 03:53:09.693008398 +0000 UTC m=+316.569071555" Sep 10 03:53:09.713106 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:09.713055 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-hf7wl" podStartSLOduration=52.783348865 podStartE2EDuration="54.71304082s" totalImagesPullingTime="1.929691955s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:52:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:53:09.712313371 +0000 UTC m=+316.588376517" watchObservedRunningTime="2026-09-10 03:53:09.71304082 +0000 UTC m=+316.589103972" Sep 10 03:53:09.891408 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:09.891368 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:53:10.651260 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:10.651217 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:10.673236 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:10.673206 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-hf7wl" Sep 10 03:53:10.753431 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:10.753396 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:10.811635 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:10.811573 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:10.947695 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:10.947603 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:11.113425 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:11.113393 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:11.164555 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:11.164526 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:11.644181 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:11.644153 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-65894fbfc9-v8z5g" Sep 10 03:53:11.726035 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:11.726002 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:11.945153 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:11.945044 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:12.046184 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:12.046152 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:12.146231 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:12.146199 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:12.247345 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:12.247249 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:12.347851 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:12.347816 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:12.467981 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:12.467945 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:12.569125 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:12.569036 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:12.671952 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:12.671920 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:19.121491 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:19.121455 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:19.182989 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:19.182952 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:23.540085 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:23.540019 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:23.595929 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:23.595891 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:26.246585 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:26.246529 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:26.301882 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:26.301832 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:26.741960 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:26.741927 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:28.549042 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:28.548999 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:33.752965 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:33.752929 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:33.853508 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:33.853464 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:36.172450 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:36.172410 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:36.224438 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:36.224401 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:37.373215 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:37.373184 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:41.233171 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:41.233109 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:41.287989 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:41.287960 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:53:52.046164 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:53:52.046085 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:54:04.891770 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:04.891726 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:04.912069 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:04.912040 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:04.928090 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:04.928058 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:07.648085 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:07.648041 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158bac16-c713-47cd-91d7-9fc36d96f570-metrics-certs\") pod \"network-metrics-daemon-2p9b8\" (UID: \"158bac16-c713-47cd-91d7-9fc36d96f570\") " pod="openshift-multus/network-metrics-daemon-2p9b8" Sep 10 03:54:07.650553 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:07.650520 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158bac16-c713-47cd-91d7-9fc36d96f570-metrics-certs\") pod \"network-metrics-daemon-2p9b8\" (UID: \"158bac16-c713-47cd-91d7-9fc36d96f570\") " pod="openshift-multus/network-metrics-daemon-2p9b8" Sep 10 03:54:07.802250 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:07.802210 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7r79s\"" Sep 10 03:54:07.810729 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:07.810698 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2p9b8" Sep 10 03:54:08.086136 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:08.086049 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2p9b8"] Sep 10 03:54:08.089698 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:54:08.089654 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod158bac16_c713_47cd_91d7_9fc36d96f570.slice/crio-8d46c5a76a4422937012fd17c11da3d316c7450cf26dd75219d4e0918e6f4c18 WatchSource:0}: Error finding container 8d46c5a76a4422937012fd17c11da3d316c7450cf26dd75219d4e0918e6f4c18: Status 404 returned error can't find the container with id 8d46c5a76a4422937012fd17c11da3d316c7450cf26dd75219d4e0918e6f4c18 Sep 10 03:54:08.917308 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:08.917258 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2p9b8" event={"ID":"158bac16-c713-47cd-91d7-9fc36d96f570","Type":"ContainerStarted","Data":"8d46c5a76a4422937012fd17c11da3d316c7450cf26dd75219d4e0918e6f4c18"} Sep 10 03:54:09.923578 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:09.923542 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2p9b8" event={"ID":"158bac16-c713-47cd-91d7-9fc36d96f570","Type":"ContainerStarted","Data":"d38385f4b10aa95b1d9689ab9717cdbab19ab6bb5ff47e03494342a7cea44cac"} Sep 10 03:54:09.923955 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:09.923604 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2p9b8" event={"ID":"158bac16-c713-47cd-91d7-9fc36d96f570","Type":"ContainerStarted","Data":"ce8f2af865e31bcce11f16129d98ceced65306189a53c60838b3ad87544a3919"} Sep 10 03:54:09.938992 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:09.938942 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-2p9b8" podStartSLOduration=374.803430935 podStartE2EDuration="6m15.938927692s" totalImagesPullingTime="1.135496757s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:47:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:54:09.937668946 +0000 UTC m=+376.813732101" watchObservedRunningTime="2026-09-10 03:54:09.938927692 +0000 UTC m=+376.814990844" Sep 10 03:54:10.021804 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:10.021768 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:54:10.082239 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:10.082208 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:54:14.421922 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:14.421873 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:54:14.482149 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:14.482114 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:54:18.672317 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:18.672260 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 03:54:18.672780 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:18.672592 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4283ea87-d273-4def-a3c7-c0a520715be0" containerName="alertmanager" containerID="cri-o://d2954efa93096a780e7fb9ecd626991a96e16db96d0afecd40964f517b5a1853" gracePeriod=120 Sep 10 03:54:18.672780 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:18.672661 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4283ea87-d273-4def-a3c7-c0a520715be0" containerName="kube-rbac-proxy-metric" containerID="cri-o://a10a681dad7099d1b357e501d3f2f1141ba626e264525743f58b432fd394b80e" gracePeriod=120 Sep 10 03:54:18.672780 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:18.672735 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4283ea87-d273-4def-a3c7-c0a520715be0" containerName="kube-rbac-proxy-web" containerID="cri-o://cc4308d7932b88c2c96f855144295e9e0788e27e328172c3b82db46794465870" gracePeriod=120 Sep 10 03:54:18.672969 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:18.672761 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4283ea87-d273-4def-a3c7-c0a520715be0" containerName="prom-label-proxy" containerID="cri-o://ed41d0ae8241eb3ade35726d4be15688f11b7875b74539680de5209076daf0cd" gracePeriod=120 Sep 10 03:54:18.672969 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:18.672741 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4283ea87-d273-4def-a3c7-c0a520715be0" containerName="kube-rbac-proxy" containerID="cri-o://76cbd3f933bfaa3c7e89b6ee2f5dc4f907352dede5256fe258bf0aa1ee0fd1c7" gracePeriod=120 Sep 10 03:54:18.672969 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:18.672923 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4283ea87-d273-4def-a3c7-c0a520715be0" containerName="config-reloader" containerID="cri-o://ece18681196ceab502eb5a239f1fb645cf5c8eac408277eafc1681e8319a795c" gracePeriod=120 Sep 10 03:54:18.962235 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:18.962143 2562 generic.go:356] "Generic (PLEG): container finished" podID="4283ea87-d273-4def-a3c7-c0a520715be0" containerID="ed41d0ae8241eb3ade35726d4be15688f11b7875b74539680de5209076daf0cd" exitCode=0 Sep 10 03:54:18.962235 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:18.962169 2562 generic.go:356] "Generic (PLEG): container finished" podID="4283ea87-d273-4def-a3c7-c0a520715be0" containerID="a10a681dad7099d1b357e501d3f2f1141ba626e264525743f58b432fd394b80e" exitCode=0 Sep 10 03:54:18.962235 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:18.962179 2562 generic.go:356] "Generic (PLEG): container finished" podID="4283ea87-d273-4def-a3c7-c0a520715be0" containerID="76cbd3f933bfaa3c7e89b6ee2f5dc4f907352dede5256fe258bf0aa1ee0fd1c7" exitCode=0 Sep 10 03:54:18.962235 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:18.962187 2562 generic.go:356] "Generic (PLEG): container finished" podID="4283ea87-d273-4def-a3c7-c0a520715be0" containerID="ece18681196ceab502eb5a239f1fb645cf5c8eac408277eafc1681e8319a795c" exitCode=0 Sep 10 03:54:18.962235 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:18.962194 2562 generic.go:356] "Generic (PLEG): container finished" podID="4283ea87-d273-4def-a3c7-c0a520715be0" containerID="d2954efa93096a780e7fb9ecd626991a96e16db96d0afecd40964f517b5a1853" exitCode=0 Sep 10 03:54:18.962235 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:18.962212 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4283ea87-d273-4def-a3c7-c0a520715be0","Type":"ContainerDied","Data":"ed41d0ae8241eb3ade35726d4be15688f11b7875b74539680de5209076daf0cd"} Sep 10 03:54:18.962554 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:18.962276 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4283ea87-d273-4def-a3c7-c0a520715be0","Type":"ContainerDied","Data":"a10a681dad7099d1b357e501d3f2f1141ba626e264525743f58b432fd394b80e"} Sep 10 03:54:18.962554 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:18.962325 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4283ea87-d273-4def-a3c7-c0a520715be0","Type":"ContainerDied","Data":"76cbd3f933bfaa3c7e89b6ee2f5dc4f907352dede5256fe258bf0aa1ee0fd1c7"} Sep 10 03:54:18.962554 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:18.962350 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4283ea87-d273-4def-a3c7-c0a520715be0","Type":"ContainerDied","Data":"ece18681196ceab502eb5a239f1fb645cf5c8eac408277eafc1681e8319a795c"} Sep 10 03:54:18.962554 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:18.962373 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4283ea87-d273-4def-a3c7-c0a520715be0","Type":"ContainerDied","Data":"d2954efa93096a780e7fb9ecd626991a96e16db96d0afecd40964f517b5a1853"} Sep 10 03:54:19.923095 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:19.923063 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:54:19.953340 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:19.953306 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-web-config\") pod \"4283ea87-d273-4def-a3c7-c0a520715be0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " Sep 10 03:54:19.953561 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:19.953373 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4283ea87-d273-4def-a3c7-c0a520715be0-config-out\") pod \"4283ea87-d273-4def-a3c7-c0a520715be0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " Sep 10 03:54:19.953561 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:19.953412 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4283ea87-d273-4def-a3c7-c0a520715be0-metrics-client-ca\") pod \"4283ea87-d273-4def-a3c7-c0a520715be0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " Sep 10 03:54:19.953561 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:19.953449 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-secret-alertmanager-kube-rbac-proxy\") pod \"4283ea87-d273-4def-a3c7-c0a520715be0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " Sep 10 03:54:19.953561 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:19.953493 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-cluster-tls-config\") pod \"4283ea87-d273-4def-a3c7-c0a520715be0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " Sep 10 03:54:19.953561 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:19.953523 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-secret-alertmanager-kube-rbac-proxy-web\") pod \"4283ea87-d273-4def-a3c7-c0a520715be0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " Sep 10 03:54:19.954067 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:19.953571 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-config-volume\") pod \"4283ea87-d273-4def-a3c7-c0a520715be0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " Sep 10 03:54:19.954067 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:19.953614 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4283ea87-d273-4def-a3c7-c0a520715be0-tls-assets\") pod \"4283ea87-d273-4def-a3c7-c0a520715be0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " Sep 10 03:54:19.954067 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:19.953646 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4283ea87-d273-4def-a3c7-c0a520715be0-alertmanager-main-db\") pod \"4283ea87-d273-4def-a3c7-c0a520715be0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " Sep 10 03:54:19.954067 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:19.953669 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5xqd4\" (UniqueName: \"kubernetes.io/projected/4283ea87-d273-4def-a3c7-c0a520715be0-kube-api-access-5xqd4\") pod \"4283ea87-d273-4def-a3c7-c0a520715be0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " Sep 10 03:54:19.954067 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:19.953722 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-secret-alertmanager-main-tls\") pod \"4283ea87-d273-4def-a3c7-c0a520715be0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " Sep 10 03:54:19.954067 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:19.953768 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4283ea87-d273-4def-a3c7-c0a520715be0-alertmanager-trusted-ca-bundle\") pod \"4283ea87-d273-4def-a3c7-c0a520715be0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " Sep 10 03:54:19.954067 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:19.953804 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"4283ea87-d273-4def-a3c7-c0a520715be0\" (UID: \"4283ea87-d273-4def-a3c7-c0a520715be0\") " Sep 10 03:54:19.954509 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:19.954433 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4283ea87-d273-4def-a3c7-c0a520715be0-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "4283ea87-d273-4def-a3c7-c0a520715be0" (UID: "4283ea87-d273-4def-a3c7-c0a520715be0"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 03:54:19.955661 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:19.955603 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4283ea87-d273-4def-a3c7-c0a520715be0-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "4283ea87-d273-4def-a3c7-c0a520715be0" (UID: "4283ea87-d273-4def-a3c7-c0a520715be0"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 03:54:19.955862 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:19.955676 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4283ea87-d273-4def-a3c7-c0a520715be0-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "4283ea87-d273-4def-a3c7-c0a520715be0" (UID: "4283ea87-d273-4def-a3c7-c0a520715be0"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 03:54:19.959261 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:19.959225 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4283ea87-d273-4def-a3c7-c0a520715be0-config-out" (OuterVolumeSpecName: "config-out") pod "4283ea87-d273-4def-a3c7-c0a520715be0" (UID: "4283ea87-d273-4def-a3c7-c0a520715be0"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 03:54:19.959568 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:19.959478 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-config-volume" (OuterVolumeSpecName: "config-volume") pod "4283ea87-d273-4def-a3c7-c0a520715be0" (UID: "4283ea87-d273-4def-a3c7-c0a520715be0"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 03:54:19.959917 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:19.959875 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "4283ea87-d273-4def-a3c7-c0a520715be0" (UID: "4283ea87-d273-4def-a3c7-c0a520715be0"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 03:54:19.959917 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:19.959888 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "4283ea87-d273-4def-a3c7-c0a520715be0" (UID: "4283ea87-d273-4def-a3c7-c0a520715be0"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 03:54:19.960174 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:19.960104 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "4283ea87-d273-4def-a3c7-c0a520715be0" (UID: "4283ea87-d273-4def-a3c7-c0a520715be0"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 03:54:19.960286 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:19.960262 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4283ea87-d273-4def-a3c7-c0a520715be0-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "4283ea87-d273-4def-a3c7-c0a520715be0" (UID: "4283ea87-d273-4def-a3c7-c0a520715be0"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 03:54:19.960574 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:19.960552 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "4283ea87-d273-4def-a3c7-c0a520715be0" (UID: "4283ea87-d273-4def-a3c7-c0a520715be0"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 03:54:19.962059 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:19.962030 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4283ea87-d273-4def-a3c7-c0a520715be0-kube-api-access-5xqd4" (OuterVolumeSpecName: "kube-api-access-5xqd4") pod "4283ea87-d273-4def-a3c7-c0a520715be0" (UID: "4283ea87-d273-4def-a3c7-c0a520715be0"). InnerVolumeSpecName "kube-api-access-5xqd4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 03:54:19.966241 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:19.966205 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "4283ea87-d273-4def-a3c7-c0a520715be0" (UID: "4283ea87-d273-4def-a3c7-c0a520715be0"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 03:54:19.973042 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:19.972995 2562 generic.go:356] "Generic (PLEG): container finished" podID="4283ea87-d273-4def-a3c7-c0a520715be0" containerID="cc4308d7932b88c2c96f855144295e9e0788e27e328172c3b82db46794465870" exitCode=0 Sep 10 03:54:19.973149 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:19.973056 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4283ea87-d273-4def-a3c7-c0a520715be0","Type":"ContainerDied","Data":"cc4308d7932b88c2c96f855144295e9e0788e27e328172c3b82db46794465870"} Sep 10 03:54:19.973149 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:19.973098 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 03:54:19.973259 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:19.973193 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4283ea87-d273-4def-a3c7-c0a520715be0","Type":"ContainerDied","Data":"ccfb457f587791ea2ec52743bb4af4476c1b7e8e6811d3eab00385b143c76f37"} Sep 10 03:54:19.974021 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:19.973230 2562 scope.go:117] "RemoveContainer" containerID="ed41d0ae8241eb3ade35726d4be15688f11b7875b74539680de5209076daf0cd" Sep 10 03:54:19.976052 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:19.976024 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-web-config" (OuterVolumeSpecName: "web-config") pod "4283ea87-d273-4def-a3c7-c0a520715be0" (UID: "4283ea87-d273-4def-a3c7-c0a520715be0"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 03:54:19.982541 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:19.982515 2562 scope.go:117] "RemoveContainer" containerID="a10a681dad7099d1b357e501d3f2f1141ba626e264525743f58b432fd394b80e" Sep 10 03:54:19.990711 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:19.990685 2562 scope.go:117] "RemoveContainer" containerID="76cbd3f933bfaa3c7e89b6ee2f5dc4f907352dede5256fe258bf0aa1ee0fd1c7" Sep 10 03:54:19.999792 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:19.999767 2562 scope.go:117] "RemoveContainer" containerID="cc4308d7932b88c2c96f855144295e9e0788e27e328172c3b82db46794465870" Sep 10 03:54:20.008194 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:20.008169 2562 scope.go:117] "RemoveContainer" containerID="ece18681196ceab502eb5a239f1fb645cf5c8eac408277eafc1681e8319a795c" Sep 10 03:54:20.016666 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:20.016642 2562 scope.go:117] "RemoveContainer" containerID="d2954efa93096a780e7fb9ecd626991a96e16db96d0afecd40964f517b5a1853" Sep 10 03:54:20.024712 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:20.024688 2562 scope.go:117] "RemoveContainer" containerID="f9da984ae568936fee7cea52cf8f1e00edc25334774925e65ea8f4c0719fc077" Sep 10 03:54:20.032171 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:20.032152 2562 scope.go:117] "RemoveContainer" containerID="ed41d0ae8241eb3ade35726d4be15688f11b7875b74539680de5209076daf0cd" Sep 10 03:54:20.032508 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:54:20.032486 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed41d0ae8241eb3ade35726d4be15688f11b7875b74539680de5209076daf0cd\": container with ID starting with ed41d0ae8241eb3ade35726d4be15688f11b7875b74539680de5209076daf0cd not found: ID does not exist" containerID="ed41d0ae8241eb3ade35726d4be15688f11b7875b74539680de5209076daf0cd" Sep 10 03:54:20.032563 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:20.032520 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed41d0ae8241eb3ade35726d4be15688f11b7875b74539680de5209076daf0cd"} err="failed to get container status \"ed41d0ae8241eb3ade35726d4be15688f11b7875b74539680de5209076daf0cd\": rpc error: code = NotFound desc = could not find container \"ed41d0ae8241eb3ade35726d4be15688f11b7875b74539680de5209076daf0cd\": container with ID starting with ed41d0ae8241eb3ade35726d4be15688f11b7875b74539680de5209076daf0cd not found: ID does not exist" Sep 10 03:54:20.032563 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:20.032540 2562 scope.go:117] "RemoveContainer" containerID="a10a681dad7099d1b357e501d3f2f1141ba626e264525743f58b432fd394b80e" Sep 10 03:54:20.032829 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:54:20.032809 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a10a681dad7099d1b357e501d3f2f1141ba626e264525743f58b432fd394b80e\": container with ID starting with a10a681dad7099d1b357e501d3f2f1141ba626e264525743f58b432fd394b80e not found: ID does not exist" containerID="a10a681dad7099d1b357e501d3f2f1141ba626e264525743f58b432fd394b80e" Sep 10 03:54:20.032898 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:20.032834 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a10a681dad7099d1b357e501d3f2f1141ba626e264525743f58b432fd394b80e"} err="failed to get container status \"a10a681dad7099d1b357e501d3f2f1141ba626e264525743f58b432fd394b80e\": rpc error: code = NotFound desc = could not find container \"a10a681dad7099d1b357e501d3f2f1141ba626e264525743f58b432fd394b80e\": container with ID starting with a10a681dad7099d1b357e501d3f2f1141ba626e264525743f58b432fd394b80e not found: ID does not exist" Sep 10 03:54:20.032898 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:20.032849 2562 scope.go:117] "RemoveContainer" containerID="76cbd3f933bfaa3c7e89b6ee2f5dc4f907352dede5256fe258bf0aa1ee0fd1c7" Sep 10 03:54:20.033094 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:54:20.033077 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76cbd3f933bfaa3c7e89b6ee2f5dc4f907352dede5256fe258bf0aa1ee0fd1c7\": container with ID starting with 76cbd3f933bfaa3c7e89b6ee2f5dc4f907352dede5256fe258bf0aa1ee0fd1c7 not found: ID does not exist" containerID="76cbd3f933bfaa3c7e89b6ee2f5dc4f907352dede5256fe258bf0aa1ee0fd1c7" Sep 10 03:54:20.033134 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:20.033098 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76cbd3f933bfaa3c7e89b6ee2f5dc4f907352dede5256fe258bf0aa1ee0fd1c7"} err="failed to get container status \"76cbd3f933bfaa3c7e89b6ee2f5dc4f907352dede5256fe258bf0aa1ee0fd1c7\": rpc error: code = NotFound desc = could not find container \"76cbd3f933bfaa3c7e89b6ee2f5dc4f907352dede5256fe258bf0aa1ee0fd1c7\": container with ID starting with 76cbd3f933bfaa3c7e89b6ee2f5dc4f907352dede5256fe258bf0aa1ee0fd1c7 not found: ID does not exist" Sep 10 03:54:20.033134 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:20.033111 2562 scope.go:117] "RemoveContainer" containerID="cc4308d7932b88c2c96f855144295e9e0788e27e328172c3b82db46794465870" Sep 10 03:54:20.033375 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:54:20.033356 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc4308d7932b88c2c96f855144295e9e0788e27e328172c3b82db46794465870\": container with ID starting with cc4308d7932b88c2c96f855144295e9e0788e27e328172c3b82db46794465870 not found: ID does not exist" containerID="cc4308d7932b88c2c96f855144295e9e0788e27e328172c3b82db46794465870" Sep 10 03:54:20.033432 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:20.033381 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc4308d7932b88c2c96f855144295e9e0788e27e328172c3b82db46794465870"} err="failed to get container status \"cc4308d7932b88c2c96f855144295e9e0788e27e328172c3b82db46794465870\": rpc error: code = NotFound desc = could not find container \"cc4308d7932b88c2c96f855144295e9e0788e27e328172c3b82db46794465870\": container with ID starting with cc4308d7932b88c2c96f855144295e9e0788e27e328172c3b82db46794465870 not found: ID does not exist" Sep 10 03:54:20.033432 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:20.033395 2562 scope.go:117] "RemoveContainer" containerID="ece18681196ceab502eb5a239f1fb645cf5c8eac408277eafc1681e8319a795c" Sep 10 03:54:20.033636 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:54:20.033621 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ece18681196ceab502eb5a239f1fb645cf5c8eac408277eafc1681e8319a795c\": container with ID starting with ece18681196ceab502eb5a239f1fb645cf5c8eac408277eafc1681e8319a795c not found: ID does not exist" containerID="ece18681196ceab502eb5a239f1fb645cf5c8eac408277eafc1681e8319a795c" Sep 10 03:54:20.033676 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:20.033639 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ece18681196ceab502eb5a239f1fb645cf5c8eac408277eafc1681e8319a795c"} err="failed to get container status \"ece18681196ceab502eb5a239f1fb645cf5c8eac408277eafc1681e8319a795c\": rpc error: code = NotFound desc = could not find container \"ece18681196ceab502eb5a239f1fb645cf5c8eac408277eafc1681e8319a795c\": container with ID starting with ece18681196ceab502eb5a239f1fb645cf5c8eac408277eafc1681e8319a795c not found: ID does not exist" Sep 10 03:54:20.033676 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:20.033650 2562 scope.go:117] "RemoveContainer" containerID="d2954efa93096a780e7fb9ecd626991a96e16db96d0afecd40964f517b5a1853" Sep 10 03:54:20.033896 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:54:20.033874 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2954efa93096a780e7fb9ecd626991a96e16db96d0afecd40964f517b5a1853\": container with ID starting with d2954efa93096a780e7fb9ecd626991a96e16db96d0afecd40964f517b5a1853 not found: ID does not exist" containerID="d2954efa93096a780e7fb9ecd626991a96e16db96d0afecd40964f517b5a1853" Sep 10 03:54:20.033937 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:20.033899 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2954efa93096a780e7fb9ecd626991a96e16db96d0afecd40964f517b5a1853"} err="failed to get container status \"d2954efa93096a780e7fb9ecd626991a96e16db96d0afecd40964f517b5a1853\": rpc error: code = NotFound desc = could not find container \"d2954efa93096a780e7fb9ecd626991a96e16db96d0afecd40964f517b5a1853\": container with ID starting with d2954efa93096a780e7fb9ecd626991a96e16db96d0afecd40964f517b5a1853 not found: ID does not exist" Sep 10 03:54:20.033937 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:20.033910 2562 scope.go:117] "RemoveContainer" containerID="f9da984ae568936fee7cea52cf8f1e00edc25334774925e65ea8f4c0719fc077" Sep 10 03:54:20.034130 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:54:20.034115 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9da984ae568936fee7cea52cf8f1e00edc25334774925e65ea8f4c0719fc077\": container with ID starting with f9da984ae568936fee7cea52cf8f1e00edc25334774925e65ea8f4c0719fc077 not found: ID does not exist" containerID="f9da984ae568936fee7cea52cf8f1e00edc25334774925e65ea8f4c0719fc077" Sep 10 03:54:20.034178 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:20.034131 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9da984ae568936fee7cea52cf8f1e00edc25334774925e65ea8f4c0719fc077"} err="failed to get container status \"f9da984ae568936fee7cea52cf8f1e00edc25334774925e65ea8f4c0719fc077\": rpc error: code = NotFound desc = could not find container \"f9da984ae568936fee7cea52cf8f1e00edc25334774925e65ea8f4c0719fc077\": container with ID starting with f9da984ae568936fee7cea52cf8f1e00edc25334774925e65ea8f4c0719fc077 not found: ID does not exist" Sep 10 03:54:20.055210 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:20.055176 2562 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4283ea87-d273-4def-a3c7-c0a520715be0-tls-assets\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:54:20.055210 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:20.055205 2562 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4283ea87-d273-4def-a3c7-c0a520715be0-alertmanager-main-db\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:54:20.055210 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:20.055215 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5xqd4\" (UniqueName: \"kubernetes.io/projected/4283ea87-d273-4def-a3c7-c0a520715be0-kube-api-access-5xqd4\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:54:20.055423 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:20.055223 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-secret-alertmanager-main-tls\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:54:20.055423 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:20.055234 2562 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4283ea87-d273-4def-a3c7-c0a520715be0-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:54:20.055423 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:20.055244 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:54:20.055423 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:20.055271 2562 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-web-config\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:54:20.055423 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:20.055279 2562 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4283ea87-d273-4def-a3c7-c0a520715be0-config-out\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:54:20.055423 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:20.055287 2562 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4283ea87-d273-4def-a3c7-c0a520715be0-metrics-client-ca\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:54:20.055423 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:20.055317 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:54:20.055423 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:20.055326 2562 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-cluster-tls-config\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:54:20.055423 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:20.055334 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:54:20.055423 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:20.055342 2562 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4283ea87-d273-4def-a3c7-c0a520715be0-config-volume\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:54:20.296934 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:20.296895 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 03:54:20.300387 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:20.300356 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 03:54:21.609968 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:21.609926 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4283ea87-d273-4def-a3c7-c0a520715be0" path="/var/lib/kubelet/pods/4283ea87-d273-4def-a3c7-c0a520715be0/volumes" Sep 10 03:54:22.858924 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:22.858881 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:54:22.929037 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:22.929005 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 03:54:22.929366 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:22.929328 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cfe42449-c4ae-4b63-be11-9f75095ed0a9" containerName="prometheus" containerID="cri-o://428b753d61571c4dc502dd71c4f11f23a758bf525c5ef1ffa75d8c400a41b43f" gracePeriod=600 Sep 10 03:54:22.929527 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:22.929359 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cfe42449-c4ae-4b63-be11-9f75095ed0a9" containerName="thanos-sidecar" containerID="cri-o://577b606f959b5114d614e5da30f1888c339ee14eed0faa21a8843effdd4bdf71" gracePeriod=600 Sep 10 03:54:22.929527 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:22.929345 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cfe42449-c4ae-4b63-be11-9f75095ed0a9" containerName="kube-rbac-proxy" containerID="cri-o://2961ec8c8dbbd6bb6227265d5fa1cef4e745facbd5e7bf664f48835d18947761" gracePeriod=600 Sep 10 03:54:22.929527 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:22.929374 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cfe42449-c4ae-4b63-be11-9f75095ed0a9" containerName="kube-rbac-proxy-thanos" containerID="cri-o://b71778daae530b59ea7d814ad711c5ac7fe4cc44ce3ef9c5b7d82acfd7497577" gracePeriod=600 Sep 10 03:54:22.929527 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:22.929363 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cfe42449-c4ae-4b63-be11-9f75095ed0a9" containerName="kube-rbac-proxy-web" containerID="cri-o://0e7b980e97aa652270deb6f7744f9f4a01aede748e0ab96c199e82cf2c500340" gracePeriod=600 Sep 10 03:54:22.929527 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:22.929385 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cfe42449-c4ae-4b63-be11-9f75095ed0a9" containerName="config-reloader" containerID="cri-o://202f84e17a4a55ac27d773c3fdf57add67a187b0c0deb43526a70936d16fed22" gracePeriod=600 Sep 10 03:54:22.931097 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:22.931053 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:54:23.180960 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.180931 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:23.280893 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.280854 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t446c\" (UniqueName: \"kubernetes.io/projected/cfe42449-c4ae-4b63-be11-9f75095ed0a9-kube-api-access-t446c\") pod \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " Sep 10 03:54:23.280893 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.280898 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-prometheus-k8s-rulefiles-1\") pod \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " Sep 10 03:54:23.281105 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.280918 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-prometheus-trusted-ca-bundle\") pod \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " Sep 10 03:54:23.281105 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.280939 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cfe42449-c4ae-4b63-be11-9f75095ed0a9-tls-assets\") pod \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " Sep 10 03:54:23.281105 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.280957 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-secret-prometheus-k8s-tls\") pod \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " Sep 10 03:54:23.281268 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.281102 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-configmap-kubelet-serving-ca-bundle\") pod \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " Sep 10 03:54:23.281268 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.281144 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-secret-kube-rbac-proxy\") pod \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " Sep 10 03:54:23.281268 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.281172 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-prometheus-k8s-rulefiles-0\") pod \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " Sep 10 03:54:23.281268 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.281196 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " Sep 10 03:54:23.281268 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.281221 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-secret-metrics-client-certs\") pod \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " Sep 10 03:54:23.281268 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.281245 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cfe42449-c4ae-4b63-be11-9f75095ed0a9-config-out\") pod \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " Sep 10 03:54:23.281583 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.281272 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-configmap-metrics-client-ca\") pod \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " Sep 10 03:54:23.281583 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.281315 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " Sep 10 03:54:23.281583 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.281326 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "cfe42449-c4ae-4b63-be11-9f75095ed0a9" (UID: "cfe42449-c4ae-4b63-be11-9f75095ed0a9"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 03:54:23.281583 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.281374 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cfe42449-c4ae-4b63-be11-9f75095ed0a9-prometheus-k8s-db\") pod \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " Sep 10 03:54:23.281583 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.281410 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-web-config\") pod \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " Sep 10 03:54:23.281583 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.281441 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-secret-grpc-tls\") pod \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " Sep 10 03:54:23.281583 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.281475 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-prometheus-k8s-rulefiles-2\") pod \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " Sep 10 03:54:23.281583 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.281509 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-config\") pod \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " Sep 10 03:54:23.281583 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.281567 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-configmap-serving-certs-ca-bundle\") pod \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " Sep 10 03:54:23.282018 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.281604 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-thanos-prometheus-http-client-file\") pod \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\" (UID: \"cfe42449-c4ae-4b63-be11-9f75095ed0a9\") " Sep 10 03:54:23.282018 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.281889 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:54:23.283331 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.281371 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "cfe42449-c4ae-4b63-be11-9f75095ed0a9" (UID: "cfe42449-c4ae-4b63-be11-9f75095ed0a9"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 03:54:23.283470 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.281891 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "cfe42449-c4ae-4b63-be11-9f75095ed0a9" (UID: "cfe42449-c4ae-4b63-be11-9f75095ed0a9"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 03:54:23.283470 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.282675 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "cfe42449-c4ae-4b63-be11-9f75095ed0a9" (UID: "cfe42449-c4ae-4b63-be11-9f75095ed0a9"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 03:54:23.283470 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.282932 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "cfe42449-c4ae-4b63-be11-9f75095ed0a9" (UID: "cfe42449-c4ae-4b63-be11-9f75095ed0a9"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 03:54:23.284469 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.283768 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfe42449-c4ae-4b63-be11-9f75095ed0a9-kube-api-access-t446c" (OuterVolumeSpecName: "kube-api-access-t446c") pod "cfe42449-c4ae-4b63-be11-9f75095ed0a9" (UID: "cfe42449-c4ae-4b63-be11-9f75095ed0a9"). InnerVolumeSpecName "kube-api-access-t446c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 03:54:23.284469 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.284352 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "cfe42449-c4ae-4b63-be11-9f75095ed0a9" (UID: "cfe42449-c4ae-4b63-be11-9f75095ed0a9"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 03:54:23.284660 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.284608 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfe42449-c4ae-4b63-be11-9f75095ed0a9-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "cfe42449-c4ae-4b63-be11-9f75095ed0a9" (UID: "cfe42449-c4ae-4b63-be11-9f75095ed0a9"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 03:54:23.284660 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.284643 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "cfe42449-c4ae-4b63-be11-9f75095ed0a9" (UID: "cfe42449-c4ae-4b63-be11-9f75095ed0a9"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 03:54:23.284903 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.284874 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "cfe42449-c4ae-4b63-be11-9f75095ed0a9" (UID: "cfe42449-c4ae-4b63-be11-9f75095ed0a9"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 03:54:23.284998 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.284919 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "cfe42449-c4ae-4b63-be11-9f75095ed0a9" (UID: "cfe42449-c4ae-4b63-be11-9f75095ed0a9"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 03:54:23.285061 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.285021 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfe42449-c4ae-4b63-be11-9f75095ed0a9-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "cfe42449-c4ae-4b63-be11-9f75095ed0a9" (UID: "cfe42449-c4ae-4b63-be11-9f75095ed0a9"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 03:54:23.285137 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.285114 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfe42449-c4ae-4b63-be11-9f75095ed0a9-config-out" (OuterVolumeSpecName: "config-out") pod "cfe42449-c4ae-4b63-be11-9f75095ed0a9" (UID: "cfe42449-c4ae-4b63-be11-9f75095ed0a9"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 03:54:23.285228 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.285131 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "cfe42449-c4ae-4b63-be11-9f75095ed0a9" (UID: "cfe42449-c4ae-4b63-be11-9f75095ed0a9"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 03:54:23.285925 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.285899 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "cfe42449-c4ae-4b63-be11-9f75095ed0a9" (UID: "cfe42449-c4ae-4b63-be11-9f75095ed0a9"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 03:54:23.286265 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.286243 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "cfe42449-c4ae-4b63-be11-9f75095ed0a9" (UID: "cfe42449-c4ae-4b63-be11-9f75095ed0a9"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 03:54:23.286648 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.286626 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "cfe42449-c4ae-4b63-be11-9f75095ed0a9" (UID: "cfe42449-c4ae-4b63-be11-9f75095ed0a9"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 03:54:23.286929 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.286907 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "cfe42449-c4ae-4b63-be11-9f75095ed0a9" (UID: "cfe42449-c4ae-4b63-be11-9f75095ed0a9"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 03:54:23.287107 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.287090 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-config" (OuterVolumeSpecName: "config") pod "cfe42449-c4ae-4b63-be11-9f75095ed0a9" (UID: "cfe42449-c4ae-4b63-be11-9f75095ed0a9"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 03:54:23.295274 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.295239 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-web-config" (OuterVolumeSpecName: "web-config") pod "cfe42449-c4ae-4b63-be11-9f75095ed0a9" (UID: "cfe42449-c4ae-4b63-be11-9f75095ed0a9"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 03:54:23.344872 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.344831 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:54:23.382669 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.382635 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cfe42449-c4ae-4b63-be11-9f75095ed0a9-prometheus-k8s-db\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:54:23.382669 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.382666 2562 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-web-config\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:54:23.382843 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.382676 2562 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-secret-grpc-tls\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:54:23.382843 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.382686 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:54:23.382843 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.382695 2562 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-config\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:54:23.382843 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.382706 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:54:23.382843 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.382716 2562 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-thanos-prometheus-http-client-file\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:54:23.382843 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.382725 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t446c\" (UniqueName: \"kubernetes.io/projected/cfe42449-c4ae-4b63-be11-9f75095ed0a9-kube-api-access-t446c\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:54:23.382843 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.382733 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-prometheus-trusted-ca-bundle\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:54:23.382843 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.382741 2562 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cfe42449-c4ae-4b63-be11-9f75095ed0a9-tls-assets\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:54:23.382843 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.382751 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-secret-prometheus-k8s-tls\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:54:23.382843 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.382761 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:54:23.382843 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.382770 2562 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-secret-kube-rbac-proxy\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:54:23.382843 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.382778 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:54:23.382843 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.382787 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:54:23.382843 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.382797 2562 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-secret-metrics-client-certs\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:54:23.382843 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.382805 2562 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cfe42449-c4ae-4b63-be11-9f75095ed0a9-config-out\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:54:23.382843 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.382813 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cfe42449-c4ae-4b63-be11-9f75095ed0a9-configmap-metrics-client-ca\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:54:23.382843 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.382822 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cfe42449-c4ae-4b63-be11-9f75095ed0a9-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:54:23.444882 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.444790 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:54:23.995236 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.995192 2562 generic.go:356] "Generic (PLEG): container finished" podID="cfe42449-c4ae-4b63-be11-9f75095ed0a9" containerID="b71778daae530b59ea7d814ad711c5ac7fe4cc44ce3ef9c5b7d82acfd7497577" exitCode=0 Sep 10 03:54:23.995236 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.995219 2562 generic.go:356] "Generic (PLEG): container finished" podID="cfe42449-c4ae-4b63-be11-9f75095ed0a9" containerID="2961ec8c8dbbd6bb6227265d5fa1cef4e745facbd5e7bf664f48835d18947761" exitCode=0 Sep 10 03:54:23.995236 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.995228 2562 generic.go:356] "Generic (PLEG): container finished" podID="cfe42449-c4ae-4b63-be11-9f75095ed0a9" containerID="0e7b980e97aa652270deb6f7744f9f4a01aede748e0ab96c199e82cf2c500340" exitCode=0 Sep 10 03:54:23.995236 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.995239 2562 generic.go:356] "Generic (PLEG): container finished" podID="cfe42449-c4ae-4b63-be11-9f75095ed0a9" containerID="577b606f959b5114d614e5da30f1888c339ee14eed0faa21a8843effdd4bdf71" exitCode=0 Sep 10 03:54:23.995236 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.995245 2562 generic.go:356] "Generic (PLEG): container finished" podID="cfe42449-c4ae-4b63-be11-9f75095ed0a9" containerID="202f84e17a4a55ac27d773c3fdf57add67a187b0c0deb43526a70936d16fed22" exitCode=0 Sep 10 03:54:23.995848 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.995255 2562 generic.go:356] "Generic (PLEG): container finished" podID="cfe42449-c4ae-4b63-be11-9f75095ed0a9" containerID="428b753d61571c4dc502dd71c4f11f23a758bf525c5ef1ffa75d8c400a41b43f" exitCode=0 Sep 10 03:54:23.995848 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.995278 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:23.995848 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.995275 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfe42449-c4ae-4b63-be11-9f75095ed0a9","Type":"ContainerDied","Data":"b71778daae530b59ea7d814ad711c5ac7fe4cc44ce3ef9c5b7d82acfd7497577"} Sep 10 03:54:23.995848 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.995379 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfe42449-c4ae-4b63-be11-9f75095ed0a9","Type":"ContainerDied","Data":"2961ec8c8dbbd6bb6227265d5fa1cef4e745facbd5e7bf664f48835d18947761"} Sep 10 03:54:23.995848 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.995424 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfe42449-c4ae-4b63-be11-9f75095ed0a9","Type":"ContainerDied","Data":"0e7b980e97aa652270deb6f7744f9f4a01aede748e0ab96c199e82cf2c500340"} Sep 10 03:54:23.995848 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.995449 2562 scope.go:117] "RemoveContainer" containerID="b71778daae530b59ea7d814ad711c5ac7fe4cc44ce3ef9c5b7d82acfd7497577" Sep 10 03:54:23.995848 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.995466 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfe42449-c4ae-4b63-be11-9f75095ed0a9","Type":"ContainerDied","Data":"577b606f959b5114d614e5da30f1888c339ee14eed0faa21a8843effdd4bdf71"} Sep 10 03:54:23.995848 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.995509 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfe42449-c4ae-4b63-be11-9f75095ed0a9","Type":"ContainerDied","Data":"202f84e17a4a55ac27d773c3fdf57add67a187b0c0deb43526a70936d16fed22"} Sep 10 03:54:23.995848 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.995553 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfe42449-c4ae-4b63-be11-9f75095ed0a9","Type":"ContainerDied","Data":"428b753d61571c4dc502dd71c4f11f23a758bf525c5ef1ffa75d8c400a41b43f"} Sep 10 03:54:23.995848 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:23.995593 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfe42449-c4ae-4b63-be11-9f75095ed0a9","Type":"ContainerDied","Data":"21b9a6daaa845d473b89b2633800b85cb4a1d0437d57d485a9162457def9ced5"} Sep 10 03:54:24.004134 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.004092 2562 scope.go:117] "RemoveContainer" containerID="2961ec8c8dbbd6bb6227265d5fa1cef4e745facbd5e7bf664f48835d18947761" Sep 10 03:54:24.011649 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.011626 2562 scope.go:117] "RemoveContainer" containerID="0e7b980e97aa652270deb6f7744f9f4a01aede748e0ab96c199e82cf2c500340" Sep 10 03:54:24.019027 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.018976 2562 scope.go:117] "RemoveContainer" containerID="577b606f959b5114d614e5da30f1888c339ee14eed0faa21a8843effdd4bdf71" Sep 10 03:54:24.020051 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.020026 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 03:54:24.026388 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.026370 2562 scope.go:117] "RemoveContainer" containerID="202f84e17a4a55ac27d773c3fdf57add67a187b0c0deb43526a70936d16fed22" Sep 10 03:54:24.031210 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.031180 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 03:54:24.033943 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.033923 2562 scope.go:117] "RemoveContainer" containerID="428b753d61571c4dc502dd71c4f11f23a758bf525c5ef1ffa75d8c400a41b43f" Sep 10 03:54:24.041675 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.041657 2562 scope.go:117] "RemoveContainer" containerID="4577130ab15a0eb7bc6795d63610134a63fc6d62972e2cb8dcefec05ba3a1814" Sep 10 03:54:24.048834 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.048810 2562 scope.go:117] "RemoveContainer" containerID="b71778daae530b59ea7d814ad711c5ac7fe4cc44ce3ef9c5b7d82acfd7497577" Sep 10 03:54:24.049156 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:54:24.049127 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b71778daae530b59ea7d814ad711c5ac7fe4cc44ce3ef9c5b7d82acfd7497577\": container with ID starting with b71778daae530b59ea7d814ad711c5ac7fe4cc44ce3ef9c5b7d82acfd7497577 not found: ID does not exist" containerID="b71778daae530b59ea7d814ad711c5ac7fe4cc44ce3ef9c5b7d82acfd7497577" Sep 10 03:54:24.049225 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.049166 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b71778daae530b59ea7d814ad711c5ac7fe4cc44ce3ef9c5b7d82acfd7497577"} err="failed to get container status \"b71778daae530b59ea7d814ad711c5ac7fe4cc44ce3ef9c5b7d82acfd7497577\": rpc error: code = NotFound desc = could not find container \"b71778daae530b59ea7d814ad711c5ac7fe4cc44ce3ef9c5b7d82acfd7497577\": container with ID starting with b71778daae530b59ea7d814ad711c5ac7fe4cc44ce3ef9c5b7d82acfd7497577 not found: ID does not exist" Sep 10 03:54:24.049225 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.049186 2562 scope.go:117] "RemoveContainer" containerID="2961ec8c8dbbd6bb6227265d5fa1cef4e745facbd5e7bf664f48835d18947761" Sep 10 03:54:24.049493 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:54:24.049474 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2961ec8c8dbbd6bb6227265d5fa1cef4e745facbd5e7bf664f48835d18947761\": container with ID starting with 2961ec8c8dbbd6bb6227265d5fa1cef4e745facbd5e7bf664f48835d18947761 not found: ID does not exist" containerID="2961ec8c8dbbd6bb6227265d5fa1cef4e745facbd5e7bf664f48835d18947761" Sep 10 03:54:24.049541 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.049498 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2961ec8c8dbbd6bb6227265d5fa1cef4e745facbd5e7bf664f48835d18947761"} err="failed to get container status \"2961ec8c8dbbd6bb6227265d5fa1cef4e745facbd5e7bf664f48835d18947761\": rpc error: code = NotFound desc = could not find container \"2961ec8c8dbbd6bb6227265d5fa1cef4e745facbd5e7bf664f48835d18947761\": container with ID starting with 2961ec8c8dbbd6bb6227265d5fa1cef4e745facbd5e7bf664f48835d18947761 not found: ID does not exist" Sep 10 03:54:24.049541 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.049513 2562 scope.go:117] "RemoveContainer" containerID="0e7b980e97aa652270deb6f7744f9f4a01aede748e0ab96c199e82cf2c500340" Sep 10 03:54:24.049779 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:54:24.049762 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e7b980e97aa652270deb6f7744f9f4a01aede748e0ab96c199e82cf2c500340\": container with ID starting with 0e7b980e97aa652270deb6f7744f9f4a01aede748e0ab96c199e82cf2c500340 not found: ID does not exist" containerID="0e7b980e97aa652270deb6f7744f9f4a01aede748e0ab96c199e82cf2c500340" Sep 10 03:54:24.049823 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.049786 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e7b980e97aa652270deb6f7744f9f4a01aede748e0ab96c199e82cf2c500340"} err="failed to get container status \"0e7b980e97aa652270deb6f7744f9f4a01aede748e0ab96c199e82cf2c500340\": rpc error: code = NotFound desc = could not find container \"0e7b980e97aa652270deb6f7744f9f4a01aede748e0ab96c199e82cf2c500340\": container with ID starting with 0e7b980e97aa652270deb6f7744f9f4a01aede748e0ab96c199e82cf2c500340 not found: ID does not exist" Sep 10 03:54:24.049823 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.049801 2562 scope.go:117] "RemoveContainer" containerID="577b606f959b5114d614e5da30f1888c339ee14eed0faa21a8843effdd4bdf71" Sep 10 03:54:24.050040 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:54:24.050024 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"577b606f959b5114d614e5da30f1888c339ee14eed0faa21a8843effdd4bdf71\": container with ID starting with 577b606f959b5114d614e5da30f1888c339ee14eed0faa21a8843effdd4bdf71 not found: ID does not exist" containerID="577b606f959b5114d614e5da30f1888c339ee14eed0faa21a8843effdd4bdf71" Sep 10 03:54:24.050080 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.050051 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"577b606f959b5114d614e5da30f1888c339ee14eed0faa21a8843effdd4bdf71"} err="failed to get container status \"577b606f959b5114d614e5da30f1888c339ee14eed0faa21a8843effdd4bdf71\": rpc error: code = NotFound desc = could not find container \"577b606f959b5114d614e5da30f1888c339ee14eed0faa21a8843effdd4bdf71\": container with ID starting with 577b606f959b5114d614e5da30f1888c339ee14eed0faa21a8843effdd4bdf71 not found: ID does not exist" Sep 10 03:54:24.050080 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.050066 2562 scope.go:117] "RemoveContainer" containerID="202f84e17a4a55ac27d773c3fdf57add67a187b0c0deb43526a70936d16fed22" Sep 10 03:54:24.050304 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:54:24.050271 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"202f84e17a4a55ac27d773c3fdf57add67a187b0c0deb43526a70936d16fed22\": container with ID starting with 202f84e17a4a55ac27d773c3fdf57add67a187b0c0deb43526a70936d16fed22 not found: ID does not exist" containerID="202f84e17a4a55ac27d773c3fdf57add67a187b0c0deb43526a70936d16fed22" Sep 10 03:54:24.050343 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.050309 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"202f84e17a4a55ac27d773c3fdf57add67a187b0c0deb43526a70936d16fed22"} err="failed to get container status \"202f84e17a4a55ac27d773c3fdf57add67a187b0c0deb43526a70936d16fed22\": rpc error: code = NotFound desc = could not find container \"202f84e17a4a55ac27d773c3fdf57add67a187b0c0deb43526a70936d16fed22\": container with ID starting with 202f84e17a4a55ac27d773c3fdf57add67a187b0c0deb43526a70936d16fed22 not found: ID does not exist" Sep 10 03:54:24.050343 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.050330 2562 scope.go:117] "RemoveContainer" containerID="428b753d61571c4dc502dd71c4f11f23a758bf525c5ef1ffa75d8c400a41b43f" Sep 10 03:54:24.050591 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:54:24.050575 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"428b753d61571c4dc502dd71c4f11f23a758bf525c5ef1ffa75d8c400a41b43f\": container with ID starting with 428b753d61571c4dc502dd71c4f11f23a758bf525c5ef1ffa75d8c400a41b43f not found: ID does not exist" containerID="428b753d61571c4dc502dd71c4f11f23a758bf525c5ef1ffa75d8c400a41b43f" Sep 10 03:54:24.050640 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.050597 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"428b753d61571c4dc502dd71c4f11f23a758bf525c5ef1ffa75d8c400a41b43f"} err="failed to get container status \"428b753d61571c4dc502dd71c4f11f23a758bf525c5ef1ffa75d8c400a41b43f\": rpc error: code = NotFound desc = could not find container \"428b753d61571c4dc502dd71c4f11f23a758bf525c5ef1ffa75d8c400a41b43f\": container with ID starting with 428b753d61571c4dc502dd71c4f11f23a758bf525c5ef1ffa75d8c400a41b43f not found: ID does not exist" Sep 10 03:54:24.050640 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.050611 2562 scope.go:117] "RemoveContainer" containerID="4577130ab15a0eb7bc6795d63610134a63fc6d62972e2cb8dcefec05ba3a1814" Sep 10 03:54:24.050824 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:54:24.050808 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4577130ab15a0eb7bc6795d63610134a63fc6d62972e2cb8dcefec05ba3a1814\": container with ID starting with 4577130ab15a0eb7bc6795d63610134a63fc6d62972e2cb8dcefec05ba3a1814 not found: ID does not exist" containerID="4577130ab15a0eb7bc6795d63610134a63fc6d62972e2cb8dcefec05ba3a1814" Sep 10 03:54:24.050862 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.050830 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4577130ab15a0eb7bc6795d63610134a63fc6d62972e2cb8dcefec05ba3a1814"} err="failed to get container status \"4577130ab15a0eb7bc6795d63610134a63fc6d62972e2cb8dcefec05ba3a1814\": rpc error: code = NotFound desc = could not find container \"4577130ab15a0eb7bc6795d63610134a63fc6d62972e2cb8dcefec05ba3a1814\": container with ID starting with 4577130ab15a0eb7bc6795d63610134a63fc6d62972e2cb8dcefec05ba3a1814 not found: ID does not exist" Sep 10 03:54:24.050862 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.050844 2562 scope.go:117] "RemoveContainer" containerID="b71778daae530b59ea7d814ad711c5ac7fe4cc44ce3ef9c5b7d82acfd7497577" Sep 10 03:54:24.051087 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.051064 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b71778daae530b59ea7d814ad711c5ac7fe4cc44ce3ef9c5b7d82acfd7497577"} err="failed to get container status \"b71778daae530b59ea7d814ad711c5ac7fe4cc44ce3ef9c5b7d82acfd7497577\": rpc error: code = NotFound desc = could not find container \"b71778daae530b59ea7d814ad711c5ac7fe4cc44ce3ef9c5b7d82acfd7497577\": container with ID starting with b71778daae530b59ea7d814ad711c5ac7fe4cc44ce3ef9c5b7d82acfd7497577 not found: ID does not exist" Sep 10 03:54:24.051136 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.051088 2562 scope.go:117] "RemoveContainer" containerID="2961ec8c8dbbd6bb6227265d5fa1cef4e745facbd5e7bf664f48835d18947761" Sep 10 03:54:24.051367 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.051286 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2961ec8c8dbbd6bb6227265d5fa1cef4e745facbd5e7bf664f48835d18947761"} err="failed to get container status \"2961ec8c8dbbd6bb6227265d5fa1cef4e745facbd5e7bf664f48835d18947761\": rpc error: code = NotFound desc = could not find container \"2961ec8c8dbbd6bb6227265d5fa1cef4e745facbd5e7bf664f48835d18947761\": container with ID starting with 2961ec8c8dbbd6bb6227265d5fa1cef4e745facbd5e7bf664f48835d18947761 not found: ID does not exist" Sep 10 03:54:24.051416 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.051367 2562 scope.go:117] "RemoveContainer" containerID="0e7b980e97aa652270deb6f7744f9f4a01aede748e0ab96c199e82cf2c500340" Sep 10 03:54:24.051622 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.051603 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e7b980e97aa652270deb6f7744f9f4a01aede748e0ab96c199e82cf2c500340"} err="failed to get container status \"0e7b980e97aa652270deb6f7744f9f4a01aede748e0ab96c199e82cf2c500340\": rpc error: code = NotFound desc = could not find container \"0e7b980e97aa652270deb6f7744f9f4a01aede748e0ab96c199e82cf2c500340\": container with ID starting with 0e7b980e97aa652270deb6f7744f9f4a01aede748e0ab96c199e82cf2c500340 not found: ID does not exist" Sep 10 03:54:24.051679 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.051621 2562 scope.go:117] "RemoveContainer" containerID="577b606f959b5114d614e5da30f1888c339ee14eed0faa21a8843effdd4bdf71" Sep 10 03:54:24.051876 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.051857 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"577b606f959b5114d614e5da30f1888c339ee14eed0faa21a8843effdd4bdf71"} err="failed to get container status \"577b606f959b5114d614e5da30f1888c339ee14eed0faa21a8843effdd4bdf71\": rpc error: code = NotFound desc = could not find container \"577b606f959b5114d614e5da30f1888c339ee14eed0faa21a8843effdd4bdf71\": container with ID starting with 577b606f959b5114d614e5da30f1888c339ee14eed0faa21a8843effdd4bdf71 not found: ID does not exist" Sep 10 03:54:24.051918 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.051877 2562 scope.go:117] "RemoveContainer" containerID="202f84e17a4a55ac27d773c3fdf57add67a187b0c0deb43526a70936d16fed22" Sep 10 03:54:24.052108 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.052089 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"202f84e17a4a55ac27d773c3fdf57add67a187b0c0deb43526a70936d16fed22"} err="failed to get container status \"202f84e17a4a55ac27d773c3fdf57add67a187b0c0deb43526a70936d16fed22\": rpc error: code = NotFound desc = could not find container \"202f84e17a4a55ac27d773c3fdf57add67a187b0c0deb43526a70936d16fed22\": container with ID starting with 202f84e17a4a55ac27d773c3fdf57add67a187b0c0deb43526a70936d16fed22 not found: ID does not exist" Sep 10 03:54:24.052145 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.052117 2562 scope.go:117] "RemoveContainer" containerID="428b753d61571c4dc502dd71c4f11f23a758bf525c5ef1ffa75d8c400a41b43f" Sep 10 03:54:24.052364 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.052348 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"428b753d61571c4dc502dd71c4f11f23a758bf525c5ef1ffa75d8c400a41b43f"} err="failed to get container status \"428b753d61571c4dc502dd71c4f11f23a758bf525c5ef1ffa75d8c400a41b43f\": rpc error: code = NotFound desc = could not find container \"428b753d61571c4dc502dd71c4f11f23a758bf525c5ef1ffa75d8c400a41b43f\": container with ID starting with 428b753d61571c4dc502dd71c4f11f23a758bf525c5ef1ffa75d8c400a41b43f not found: ID does not exist" Sep 10 03:54:24.052404 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.052364 2562 scope.go:117] "RemoveContainer" containerID="4577130ab15a0eb7bc6795d63610134a63fc6d62972e2cb8dcefec05ba3a1814" Sep 10 03:54:24.052579 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.052560 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4577130ab15a0eb7bc6795d63610134a63fc6d62972e2cb8dcefec05ba3a1814"} err="failed to get container status \"4577130ab15a0eb7bc6795d63610134a63fc6d62972e2cb8dcefec05ba3a1814\": rpc error: code = NotFound desc = could not find container \"4577130ab15a0eb7bc6795d63610134a63fc6d62972e2cb8dcefec05ba3a1814\": container with ID starting with 4577130ab15a0eb7bc6795d63610134a63fc6d62972e2cb8dcefec05ba3a1814 not found: ID does not exist" Sep 10 03:54:24.052627 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.052579 2562 scope.go:117] "RemoveContainer" containerID="b71778daae530b59ea7d814ad711c5ac7fe4cc44ce3ef9c5b7d82acfd7497577" Sep 10 03:54:24.052800 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.052783 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b71778daae530b59ea7d814ad711c5ac7fe4cc44ce3ef9c5b7d82acfd7497577"} err="failed to get container status \"b71778daae530b59ea7d814ad711c5ac7fe4cc44ce3ef9c5b7d82acfd7497577\": rpc error: code = NotFound desc = could not find container \"b71778daae530b59ea7d814ad711c5ac7fe4cc44ce3ef9c5b7d82acfd7497577\": container with ID starting with b71778daae530b59ea7d814ad711c5ac7fe4cc44ce3ef9c5b7d82acfd7497577 not found: ID does not exist" Sep 10 03:54:24.052800 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.052799 2562 scope.go:117] "RemoveContainer" containerID="2961ec8c8dbbd6bb6227265d5fa1cef4e745facbd5e7bf664f48835d18947761" Sep 10 03:54:24.053011 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.052993 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2961ec8c8dbbd6bb6227265d5fa1cef4e745facbd5e7bf664f48835d18947761"} err="failed to get container status \"2961ec8c8dbbd6bb6227265d5fa1cef4e745facbd5e7bf664f48835d18947761\": rpc error: code = NotFound desc = could not find container \"2961ec8c8dbbd6bb6227265d5fa1cef4e745facbd5e7bf664f48835d18947761\": container with ID starting with 2961ec8c8dbbd6bb6227265d5fa1cef4e745facbd5e7bf664f48835d18947761 not found: ID does not exist" Sep 10 03:54:24.053049 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.053010 2562 scope.go:117] "RemoveContainer" containerID="0e7b980e97aa652270deb6f7744f9f4a01aede748e0ab96c199e82cf2c500340" Sep 10 03:54:24.053243 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.053225 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e7b980e97aa652270deb6f7744f9f4a01aede748e0ab96c199e82cf2c500340"} err="failed to get container status \"0e7b980e97aa652270deb6f7744f9f4a01aede748e0ab96c199e82cf2c500340\": rpc error: code = NotFound desc = could not find container \"0e7b980e97aa652270deb6f7744f9f4a01aede748e0ab96c199e82cf2c500340\": container with ID starting with 0e7b980e97aa652270deb6f7744f9f4a01aede748e0ab96c199e82cf2c500340 not found: ID does not exist" Sep 10 03:54:24.053283 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.053244 2562 scope.go:117] "RemoveContainer" containerID="577b606f959b5114d614e5da30f1888c339ee14eed0faa21a8843effdd4bdf71" Sep 10 03:54:24.053462 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.053445 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"577b606f959b5114d614e5da30f1888c339ee14eed0faa21a8843effdd4bdf71"} err="failed to get container status \"577b606f959b5114d614e5da30f1888c339ee14eed0faa21a8843effdd4bdf71\": rpc error: code = NotFound desc = could not find container \"577b606f959b5114d614e5da30f1888c339ee14eed0faa21a8843effdd4bdf71\": container with ID starting with 577b606f959b5114d614e5da30f1888c339ee14eed0faa21a8843effdd4bdf71 not found: ID does not exist" Sep 10 03:54:24.053498 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.053462 2562 scope.go:117] "RemoveContainer" containerID="202f84e17a4a55ac27d773c3fdf57add67a187b0c0deb43526a70936d16fed22" Sep 10 03:54:24.053686 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.053663 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"202f84e17a4a55ac27d773c3fdf57add67a187b0c0deb43526a70936d16fed22"} err="failed to get container status \"202f84e17a4a55ac27d773c3fdf57add67a187b0c0deb43526a70936d16fed22\": rpc error: code = NotFound desc = could not find container \"202f84e17a4a55ac27d773c3fdf57add67a187b0c0deb43526a70936d16fed22\": container with ID starting with 202f84e17a4a55ac27d773c3fdf57add67a187b0c0deb43526a70936d16fed22 not found: ID does not exist" Sep 10 03:54:24.053686 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.053685 2562 scope.go:117] "RemoveContainer" containerID="428b753d61571c4dc502dd71c4f11f23a758bf525c5ef1ffa75d8c400a41b43f" Sep 10 03:54:24.053914 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.053897 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"428b753d61571c4dc502dd71c4f11f23a758bf525c5ef1ffa75d8c400a41b43f"} err="failed to get container status \"428b753d61571c4dc502dd71c4f11f23a758bf525c5ef1ffa75d8c400a41b43f\": rpc error: code = NotFound desc = could not find container \"428b753d61571c4dc502dd71c4f11f23a758bf525c5ef1ffa75d8c400a41b43f\": container with ID starting with 428b753d61571c4dc502dd71c4f11f23a758bf525c5ef1ffa75d8c400a41b43f not found: ID does not exist" Sep 10 03:54:24.053960 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.053915 2562 scope.go:117] "RemoveContainer" containerID="4577130ab15a0eb7bc6795d63610134a63fc6d62972e2cb8dcefec05ba3a1814" Sep 10 03:54:24.054133 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.054117 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4577130ab15a0eb7bc6795d63610134a63fc6d62972e2cb8dcefec05ba3a1814"} err="failed to get container status \"4577130ab15a0eb7bc6795d63610134a63fc6d62972e2cb8dcefec05ba3a1814\": rpc error: code = NotFound desc = could not find container \"4577130ab15a0eb7bc6795d63610134a63fc6d62972e2cb8dcefec05ba3a1814\": container with ID starting with 4577130ab15a0eb7bc6795d63610134a63fc6d62972e2cb8dcefec05ba3a1814 not found: ID does not exist" Sep 10 03:54:24.054172 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.054133 2562 scope.go:117] "RemoveContainer" containerID="b71778daae530b59ea7d814ad711c5ac7fe4cc44ce3ef9c5b7d82acfd7497577" Sep 10 03:54:24.054344 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.054328 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b71778daae530b59ea7d814ad711c5ac7fe4cc44ce3ef9c5b7d82acfd7497577"} err="failed to get container status \"b71778daae530b59ea7d814ad711c5ac7fe4cc44ce3ef9c5b7d82acfd7497577\": rpc error: code = NotFound desc = could not find container \"b71778daae530b59ea7d814ad711c5ac7fe4cc44ce3ef9c5b7d82acfd7497577\": container with ID starting with b71778daae530b59ea7d814ad711c5ac7fe4cc44ce3ef9c5b7d82acfd7497577 not found: ID does not exist" Sep 10 03:54:24.054395 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.054344 2562 scope.go:117] "RemoveContainer" containerID="2961ec8c8dbbd6bb6227265d5fa1cef4e745facbd5e7bf664f48835d18947761" Sep 10 03:54:24.054546 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.054530 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2961ec8c8dbbd6bb6227265d5fa1cef4e745facbd5e7bf664f48835d18947761"} err="failed to get container status \"2961ec8c8dbbd6bb6227265d5fa1cef4e745facbd5e7bf664f48835d18947761\": rpc error: code = NotFound desc = could not find container \"2961ec8c8dbbd6bb6227265d5fa1cef4e745facbd5e7bf664f48835d18947761\": container with ID starting with 2961ec8c8dbbd6bb6227265d5fa1cef4e745facbd5e7bf664f48835d18947761 not found: ID does not exist" Sep 10 03:54:24.054583 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.054545 2562 scope.go:117] "RemoveContainer" containerID="0e7b980e97aa652270deb6f7744f9f4a01aede748e0ab96c199e82cf2c500340" Sep 10 03:54:24.054764 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.054744 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e7b980e97aa652270deb6f7744f9f4a01aede748e0ab96c199e82cf2c500340"} err="failed to get container status \"0e7b980e97aa652270deb6f7744f9f4a01aede748e0ab96c199e82cf2c500340\": rpc error: code = NotFound desc = could not find container \"0e7b980e97aa652270deb6f7744f9f4a01aede748e0ab96c199e82cf2c500340\": container with ID starting with 0e7b980e97aa652270deb6f7744f9f4a01aede748e0ab96c199e82cf2c500340 not found: ID does not exist" Sep 10 03:54:24.054834 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.054766 2562 scope.go:117] "RemoveContainer" containerID="577b606f959b5114d614e5da30f1888c339ee14eed0faa21a8843effdd4bdf71" Sep 10 03:54:24.055000 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.054985 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"577b606f959b5114d614e5da30f1888c339ee14eed0faa21a8843effdd4bdf71"} err="failed to get container status \"577b606f959b5114d614e5da30f1888c339ee14eed0faa21a8843effdd4bdf71\": rpc error: code = NotFound desc = could not find container \"577b606f959b5114d614e5da30f1888c339ee14eed0faa21a8843effdd4bdf71\": container with ID starting with 577b606f959b5114d614e5da30f1888c339ee14eed0faa21a8843effdd4bdf71 not found: ID does not exist" Sep 10 03:54:24.055045 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.055000 2562 scope.go:117] "RemoveContainer" containerID="202f84e17a4a55ac27d773c3fdf57add67a187b0c0deb43526a70936d16fed22" Sep 10 03:54:24.055180 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.055162 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"202f84e17a4a55ac27d773c3fdf57add67a187b0c0deb43526a70936d16fed22"} err="failed to get container status \"202f84e17a4a55ac27d773c3fdf57add67a187b0c0deb43526a70936d16fed22\": rpc error: code = NotFound desc = could not find container \"202f84e17a4a55ac27d773c3fdf57add67a187b0c0deb43526a70936d16fed22\": container with ID starting with 202f84e17a4a55ac27d773c3fdf57add67a187b0c0deb43526a70936d16fed22 not found: ID does not exist" Sep 10 03:54:24.055228 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.055181 2562 scope.go:117] "RemoveContainer" containerID="428b753d61571c4dc502dd71c4f11f23a758bf525c5ef1ffa75d8c400a41b43f" Sep 10 03:54:24.055375 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.055359 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"428b753d61571c4dc502dd71c4f11f23a758bf525c5ef1ffa75d8c400a41b43f"} err="failed to get container status \"428b753d61571c4dc502dd71c4f11f23a758bf525c5ef1ffa75d8c400a41b43f\": rpc error: code = NotFound desc = could not find container \"428b753d61571c4dc502dd71c4f11f23a758bf525c5ef1ffa75d8c400a41b43f\": container with ID starting with 428b753d61571c4dc502dd71c4f11f23a758bf525c5ef1ffa75d8c400a41b43f not found: ID does not exist" Sep 10 03:54:24.055418 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.055374 2562 scope.go:117] "RemoveContainer" containerID="4577130ab15a0eb7bc6795d63610134a63fc6d62972e2cb8dcefec05ba3a1814" Sep 10 03:54:24.055592 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.055576 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4577130ab15a0eb7bc6795d63610134a63fc6d62972e2cb8dcefec05ba3a1814"} err="failed to get container status \"4577130ab15a0eb7bc6795d63610134a63fc6d62972e2cb8dcefec05ba3a1814\": rpc error: code = NotFound desc = could not find container \"4577130ab15a0eb7bc6795d63610134a63fc6d62972e2cb8dcefec05ba3a1814\": container with ID starting with 4577130ab15a0eb7bc6795d63610134a63fc6d62972e2cb8dcefec05ba3a1814 not found: ID does not exist" Sep 10 03:54:24.055636 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.055592 2562 scope.go:117] "RemoveContainer" containerID="b71778daae530b59ea7d814ad711c5ac7fe4cc44ce3ef9c5b7d82acfd7497577" Sep 10 03:54:24.055796 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.055777 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b71778daae530b59ea7d814ad711c5ac7fe4cc44ce3ef9c5b7d82acfd7497577"} err="failed to get container status \"b71778daae530b59ea7d814ad711c5ac7fe4cc44ce3ef9c5b7d82acfd7497577\": rpc error: code = NotFound desc = could not find container \"b71778daae530b59ea7d814ad711c5ac7fe4cc44ce3ef9c5b7d82acfd7497577\": container with ID starting with b71778daae530b59ea7d814ad711c5ac7fe4cc44ce3ef9c5b7d82acfd7497577 not found: ID does not exist" Sep 10 03:54:24.055861 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.055798 2562 scope.go:117] "RemoveContainer" containerID="2961ec8c8dbbd6bb6227265d5fa1cef4e745facbd5e7bf664f48835d18947761" Sep 10 03:54:24.056013 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.055997 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2961ec8c8dbbd6bb6227265d5fa1cef4e745facbd5e7bf664f48835d18947761"} err="failed to get container status \"2961ec8c8dbbd6bb6227265d5fa1cef4e745facbd5e7bf664f48835d18947761\": rpc error: code = NotFound desc = could not find container \"2961ec8c8dbbd6bb6227265d5fa1cef4e745facbd5e7bf664f48835d18947761\": container with ID starting with 2961ec8c8dbbd6bb6227265d5fa1cef4e745facbd5e7bf664f48835d18947761 not found: ID does not exist" Sep 10 03:54:24.056061 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.056013 2562 scope.go:117] "RemoveContainer" containerID="0e7b980e97aa652270deb6f7744f9f4a01aede748e0ab96c199e82cf2c500340" Sep 10 03:54:24.056203 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.056187 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e7b980e97aa652270deb6f7744f9f4a01aede748e0ab96c199e82cf2c500340"} err="failed to get container status \"0e7b980e97aa652270deb6f7744f9f4a01aede748e0ab96c199e82cf2c500340\": rpc error: code = NotFound desc = could not find container \"0e7b980e97aa652270deb6f7744f9f4a01aede748e0ab96c199e82cf2c500340\": container with ID starting with 0e7b980e97aa652270deb6f7744f9f4a01aede748e0ab96c199e82cf2c500340 not found: ID does not exist" Sep 10 03:54:24.056242 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.056203 2562 scope.go:117] "RemoveContainer" containerID="577b606f959b5114d614e5da30f1888c339ee14eed0faa21a8843effdd4bdf71" Sep 10 03:54:24.056420 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.056404 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"577b606f959b5114d614e5da30f1888c339ee14eed0faa21a8843effdd4bdf71"} err="failed to get container status \"577b606f959b5114d614e5da30f1888c339ee14eed0faa21a8843effdd4bdf71\": rpc error: code = NotFound desc = could not find container \"577b606f959b5114d614e5da30f1888c339ee14eed0faa21a8843effdd4bdf71\": container with ID starting with 577b606f959b5114d614e5da30f1888c339ee14eed0faa21a8843effdd4bdf71 not found: ID does not exist" Sep 10 03:54:24.056465 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.056420 2562 scope.go:117] "RemoveContainer" containerID="202f84e17a4a55ac27d773c3fdf57add67a187b0c0deb43526a70936d16fed22" Sep 10 03:54:24.056616 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.056594 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"202f84e17a4a55ac27d773c3fdf57add67a187b0c0deb43526a70936d16fed22"} err="failed to get container status \"202f84e17a4a55ac27d773c3fdf57add67a187b0c0deb43526a70936d16fed22\": rpc error: code = NotFound desc = could not find container \"202f84e17a4a55ac27d773c3fdf57add67a187b0c0deb43526a70936d16fed22\": container with ID starting with 202f84e17a4a55ac27d773c3fdf57add67a187b0c0deb43526a70936d16fed22 not found: ID does not exist" Sep 10 03:54:24.056679 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.056616 2562 scope.go:117] "RemoveContainer" containerID="428b753d61571c4dc502dd71c4f11f23a758bf525c5ef1ffa75d8c400a41b43f" Sep 10 03:54:24.056854 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.056832 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"428b753d61571c4dc502dd71c4f11f23a758bf525c5ef1ffa75d8c400a41b43f"} err="failed to get container status \"428b753d61571c4dc502dd71c4f11f23a758bf525c5ef1ffa75d8c400a41b43f\": rpc error: code = NotFound desc = could not find container \"428b753d61571c4dc502dd71c4f11f23a758bf525c5ef1ffa75d8c400a41b43f\": container with ID starting with 428b753d61571c4dc502dd71c4f11f23a758bf525c5ef1ffa75d8c400a41b43f not found: ID does not exist" Sep 10 03:54:24.056944 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.056854 2562 scope.go:117] "RemoveContainer" containerID="4577130ab15a0eb7bc6795d63610134a63fc6d62972e2cb8dcefec05ba3a1814" Sep 10 03:54:24.057084 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.057059 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4577130ab15a0eb7bc6795d63610134a63fc6d62972e2cb8dcefec05ba3a1814"} err="failed to get container status \"4577130ab15a0eb7bc6795d63610134a63fc6d62972e2cb8dcefec05ba3a1814\": rpc error: code = NotFound desc = could not find container \"4577130ab15a0eb7bc6795d63610134a63fc6d62972e2cb8dcefec05ba3a1814\": container with ID starting with 4577130ab15a0eb7bc6795d63610134a63fc6d62972e2cb8dcefec05ba3a1814 not found: ID does not exist" Sep 10 03:54:24.057084 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.057082 2562 scope.go:117] "RemoveContainer" containerID="b71778daae530b59ea7d814ad711c5ac7fe4cc44ce3ef9c5b7d82acfd7497577" Sep 10 03:54:24.057337 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.057321 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b71778daae530b59ea7d814ad711c5ac7fe4cc44ce3ef9c5b7d82acfd7497577"} err="failed to get container status \"b71778daae530b59ea7d814ad711c5ac7fe4cc44ce3ef9c5b7d82acfd7497577\": rpc error: code = NotFound desc = could not find container \"b71778daae530b59ea7d814ad711c5ac7fe4cc44ce3ef9c5b7d82acfd7497577\": container with ID starting with b71778daae530b59ea7d814ad711c5ac7fe4cc44ce3ef9c5b7d82acfd7497577 not found: ID does not exist" Sep 10 03:54:24.057337 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.057336 2562 scope.go:117] "RemoveContainer" containerID="2961ec8c8dbbd6bb6227265d5fa1cef4e745facbd5e7bf664f48835d18947761" Sep 10 03:54:24.057555 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.057536 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2961ec8c8dbbd6bb6227265d5fa1cef4e745facbd5e7bf664f48835d18947761"} err="failed to get container status \"2961ec8c8dbbd6bb6227265d5fa1cef4e745facbd5e7bf664f48835d18947761\": rpc error: code = NotFound desc = could not find container \"2961ec8c8dbbd6bb6227265d5fa1cef4e745facbd5e7bf664f48835d18947761\": container with ID starting with 2961ec8c8dbbd6bb6227265d5fa1cef4e745facbd5e7bf664f48835d18947761 not found: ID does not exist" Sep 10 03:54:24.057618 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.057557 2562 scope.go:117] "RemoveContainer" containerID="0e7b980e97aa652270deb6f7744f9f4a01aede748e0ab96c199e82cf2c500340" Sep 10 03:54:24.057824 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.057806 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e7b980e97aa652270deb6f7744f9f4a01aede748e0ab96c199e82cf2c500340"} err="failed to get container status \"0e7b980e97aa652270deb6f7744f9f4a01aede748e0ab96c199e82cf2c500340\": rpc error: code = NotFound desc = could not find container \"0e7b980e97aa652270deb6f7744f9f4a01aede748e0ab96c199e82cf2c500340\": container with ID starting with 0e7b980e97aa652270deb6f7744f9f4a01aede748e0ab96c199e82cf2c500340 not found: ID does not exist" Sep 10 03:54:24.057871 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.057827 2562 scope.go:117] "RemoveContainer" containerID="577b606f959b5114d614e5da30f1888c339ee14eed0faa21a8843effdd4bdf71" Sep 10 03:54:24.058031 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.058015 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"577b606f959b5114d614e5da30f1888c339ee14eed0faa21a8843effdd4bdf71"} err="failed to get container status \"577b606f959b5114d614e5da30f1888c339ee14eed0faa21a8843effdd4bdf71\": rpc error: code = NotFound desc = could not find container \"577b606f959b5114d614e5da30f1888c339ee14eed0faa21a8843effdd4bdf71\": container with ID starting with 577b606f959b5114d614e5da30f1888c339ee14eed0faa21a8843effdd4bdf71 not found: ID does not exist" Sep 10 03:54:24.058076 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.058032 2562 scope.go:117] "RemoveContainer" containerID="202f84e17a4a55ac27d773c3fdf57add67a187b0c0deb43526a70936d16fed22" Sep 10 03:54:24.058259 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.058244 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"202f84e17a4a55ac27d773c3fdf57add67a187b0c0deb43526a70936d16fed22"} err="failed to get container status \"202f84e17a4a55ac27d773c3fdf57add67a187b0c0deb43526a70936d16fed22\": rpc error: code = NotFound desc = could not find container \"202f84e17a4a55ac27d773c3fdf57add67a187b0c0deb43526a70936d16fed22\": container with ID starting with 202f84e17a4a55ac27d773c3fdf57add67a187b0c0deb43526a70936d16fed22 not found: ID does not exist" Sep 10 03:54:24.058315 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.058259 2562 scope.go:117] "RemoveContainer" containerID="428b753d61571c4dc502dd71c4f11f23a758bf525c5ef1ffa75d8c400a41b43f" Sep 10 03:54:24.058502 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.058484 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"428b753d61571c4dc502dd71c4f11f23a758bf525c5ef1ffa75d8c400a41b43f"} err="failed to get container status \"428b753d61571c4dc502dd71c4f11f23a758bf525c5ef1ffa75d8c400a41b43f\": rpc error: code = NotFound desc = could not find container \"428b753d61571c4dc502dd71c4f11f23a758bf525c5ef1ffa75d8c400a41b43f\": container with ID starting with 428b753d61571c4dc502dd71c4f11f23a758bf525c5ef1ffa75d8c400a41b43f not found: ID does not exist" Sep 10 03:54:24.058502 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.058502 2562 scope.go:117] "RemoveContainer" containerID="4577130ab15a0eb7bc6795d63610134a63fc6d62972e2cb8dcefec05ba3a1814" Sep 10 03:54:24.058713 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.058698 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4577130ab15a0eb7bc6795d63610134a63fc6d62972e2cb8dcefec05ba3a1814"} err="failed to get container status \"4577130ab15a0eb7bc6795d63610134a63fc6d62972e2cb8dcefec05ba3a1814\": rpc error: code = NotFound desc = could not find container \"4577130ab15a0eb7bc6795d63610134a63fc6d62972e2cb8dcefec05ba3a1814\": container with ID starting with 4577130ab15a0eb7bc6795d63610134a63fc6d62972e2cb8dcefec05ba3a1814 not found: ID does not exist" Sep 10 03:54:24.081930 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.081889 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 03:54:24.082208 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082194 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4283ea87-d273-4def-a3c7-c0a520715be0" containerName="config-reloader" Sep 10 03:54:24.082250 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082210 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4283ea87-d273-4def-a3c7-c0a520715be0" containerName="config-reloader" Sep 10 03:54:24.082250 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082221 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4283ea87-d273-4def-a3c7-c0a520715be0" containerName="alertmanager" Sep 10 03:54:24.082250 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082230 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4283ea87-d273-4def-a3c7-c0a520715be0" containerName="alertmanager" Sep 10 03:54:24.082250 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082239 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4283ea87-d273-4def-a3c7-c0a520715be0" containerName="kube-rbac-proxy-web" Sep 10 03:54:24.082250 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082246 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4283ea87-d273-4def-a3c7-c0a520715be0" containerName="kube-rbac-proxy-web" Sep 10 03:54:24.082419 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082253 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4283ea87-d273-4def-a3c7-c0a520715be0" containerName="kube-rbac-proxy-metric" Sep 10 03:54:24.082419 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082258 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4283ea87-d273-4def-a3c7-c0a520715be0" containerName="kube-rbac-proxy-metric" Sep 10 03:54:24.082419 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082267 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfe42449-c4ae-4b63-be11-9f75095ed0a9" containerName="thanos-sidecar" Sep 10 03:54:24.082419 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082272 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfe42449-c4ae-4b63-be11-9f75095ed0a9" containerName="thanos-sidecar" Sep 10 03:54:24.082419 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082281 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfe42449-c4ae-4b63-be11-9f75095ed0a9" containerName="kube-rbac-proxy-web" Sep 10 03:54:24.082419 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082286 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfe42449-c4ae-4b63-be11-9f75095ed0a9" containerName="kube-rbac-proxy-web" Sep 10 03:54:24.082419 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082309 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfe42449-c4ae-4b63-be11-9f75095ed0a9" containerName="kube-rbac-proxy" Sep 10 03:54:24.082419 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082317 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfe42449-c4ae-4b63-be11-9f75095ed0a9" containerName="kube-rbac-proxy" Sep 10 03:54:24.082419 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082332 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfe42449-c4ae-4b63-be11-9f75095ed0a9" containerName="kube-rbac-proxy-thanos" Sep 10 03:54:24.082419 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082339 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfe42449-c4ae-4b63-be11-9f75095ed0a9" containerName="kube-rbac-proxy-thanos" Sep 10 03:54:24.082419 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082349 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfe42449-c4ae-4b63-be11-9f75095ed0a9" containerName="prometheus" Sep 10 03:54:24.082419 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082355 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfe42449-c4ae-4b63-be11-9f75095ed0a9" containerName="prometheus" Sep 10 03:54:24.082419 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082363 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4283ea87-d273-4def-a3c7-c0a520715be0" containerName="init-config-reloader" Sep 10 03:54:24.082419 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082368 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4283ea87-d273-4def-a3c7-c0a520715be0" containerName="init-config-reloader" Sep 10 03:54:24.082419 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082373 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4283ea87-d273-4def-a3c7-c0a520715be0" containerName="kube-rbac-proxy" Sep 10 03:54:24.082419 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082378 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4283ea87-d273-4def-a3c7-c0a520715be0" containerName="kube-rbac-proxy" Sep 10 03:54:24.082419 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082387 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfe42449-c4ae-4b63-be11-9f75095ed0a9" containerName="config-reloader" Sep 10 03:54:24.082419 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082395 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfe42449-c4ae-4b63-be11-9f75095ed0a9" containerName="config-reloader" Sep 10 03:54:24.082419 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082401 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ba38a7e-088d-4fdf-840c-c85b7826f1a6" containerName="registry" Sep 10 03:54:24.082419 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082406 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ba38a7e-088d-4fdf-840c-c85b7826f1a6" containerName="registry" Sep 10 03:54:24.082419 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082416 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4283ea87-d273-4def-a3c7-c0a520715be0" containerName="prom-label-proxy" Sep 10 03:54:24.082419 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082421 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4283ea87-d273-4def-a3c7-c0a520715be0" containerName="prom-label-proxy" Sep 10 03:54:24.082419 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082428 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfe42449-c4ae-4b63-be11-9f75095ed0a9" containerName="init-config-reloader" Sep 10 03:54:24.083020 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082433 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfe42449-c4ae-4b63-be11-9f75095ed0a9" containerName="init-config-reloader" Sep 10 03:54:24.083020 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082485 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4283ea87-d273-4def-a3c7-c0a520715be0" containerName="prom-label-proxy" Sep 10 03:54:24.083020 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082494 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ba38a7e-088d-4fdf-840c-c85b7826f1a6" containerName="registry" Sep 10 03:54:24.083020 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082499 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfe42449-c4ae-4b63-be11-9f75095ed0a9" containerName="thanos-sidecar" Sep 10 03:54:24.083020 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082505 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4283ea87-d273-4def-a3c7-c0a520715be0" containerName="config-reloader" Sep 10 03:54:24.083020 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082510 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfe42449-c4ae-4b63-be11-9f75095ed0a9" containerName="config-reloader" Sep 10 03:54:24.083020 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082516 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfe42449-c4ae-4b63-be11-9f75095ed0a9" containerName="prometheus" Sep 10 03:54:24.083020 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082521 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfe42449-c4ae-4b63-be11-9f75095ed0a9" containerName="kube-rbac-proxy" Sep 10 03:54:24.083020 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082526 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4283ea87-d273-4def-a3c7-c0a520715be0" containerName="alertmanager" Sep 10 03:54:24.083020 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082532 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfe42449-c4ae-4b63-be11-9f75095ed0a9" containerName="kube-rbac-proxy-thanos" Sep 10 03:54:24.083020 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082538 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4283ea87-d273-4def-a3c7-c0a520715be0" containerName="kube-rbac-proxy" Sep 10 03:54:24.083020 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082545 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4283ea87-d273-4def-a3c7-c0a520715be0" containerName="kube-rbac-proxy-metric" Sep 10 03:54:24.083020 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082553 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfe42449-c4ae-4b63-be11-9f75095ed0a9" containerName="kube-rbac-proxy-web" Sep 10 03:54:24.083020 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.082558 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4283ea87-d273-4def-a3c7-c0a520715be0" containerName="kube-rbac-proxy-web" Sep 10 03:54:24.090085 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.090056 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.092585 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.092546 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 10 03:54:24.092723 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.092623 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-lnc3oa1mqbvh\"" Sep 10 03:54:24.092977 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.092938 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 10 03:54:24.092977 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.092965 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-p26td\"" Sep 10 03:54:24.093159 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.092997 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 10 03:54:24.093159 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.093000 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 10 03:54:24.093159 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.093104 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 10 03:54:24.093159 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.093120 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 10 03:54:24.093159 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.093107 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 10 03:54:24.093430 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.093328 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 10 03:54:24.093665 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.093647 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 10 03:54:24.093770 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.093676 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 10 03:54:24.093957 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.093939 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 10 03:54:24.094079 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.094056 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 10 03:54:24.094258 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.094244 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 10 03:54:24.096893 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.096875 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 10 03:54:24.098924 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.098903 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 10 03:54:24.104363 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.104336 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 03:54:24.188850 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.188808 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/59c7afc9-5012-4dae-8cea-2575633ab869-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.188850 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.188850 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/59c7afc9-5012-4dae-8cea-2575633ab869-config-out\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.189075 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.188874 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/59c7afc9-5012-4dae-8cea-2575633ab869-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.189075 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.188893 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/59c7afc9-5012-4dae-8cea-2575633ab869-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.189075 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.188911 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/59c7afc9-5012-4dae-8cea-2575633ab869-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.189075 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.188926 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/59c7afc9-5012-4dae-8cea-2575633ab869-web-config\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.189075 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.188984 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/59c7afc9-5012-4dae-8cea-2575633ab869-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.189075 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.189029 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/59c7afc9-5012-4dae-8cea-2575633ab869-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.189075 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.189049 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/59c7afc9-5012-4dae-8cea-2575633ab869-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.189075 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.189067 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/59c7afc9-5012-4dae-8cea-2575633ab869-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.189413 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.189083 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/59c7afc9-5012-4dae-8cea-2575633ab869-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.189413 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.189107 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/59c7afc9-5012-4dae-8cea-2575633ab869-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.189413 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.189143 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59c7afc9-5012-4dae-8cea-2575633ab869-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.189413 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.189183 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/59c7afc9-5012-4dae-8cea-2575633ab869-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.189413 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.189204 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/59c7afc9-5012-4dae-8cea-2575633ab869-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.189413 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.189216 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5gc8\" (UniqueName: \"kubernetes.io/projected/59c7afc9-5012-4dae-8cea-2575633ab869-kube-api-access-z5gc8\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.189413 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.189236 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/59c7afc9-5012-4dae-8cea-2575633ab869-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.189413 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.189252 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59c7afc9-5012-4dae-8cea-2575633ab869-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.189413 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.189267 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/59c7afc9-5012-4dae-8cea-2575633ab869-config\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.189413 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.189355 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59c7afc9-5012-4dae-8cea-2575633ab869-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.290750 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.290642 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/59c7afc9-5012-4dae-8cea-2575633ab869-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.290750 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.290693 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/59c7afc9-5012-4dae-8cea-2575633ab869-web-config\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.290750 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.290724 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/59c7afc9-5012-4dae-8cea-2575633ab869-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.290750 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.290741 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/59c7afc9-5012-4dae-8cea-2575633ab869-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.290750 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.290756 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/59c7afc9-5012-4dae-8cea-2575633ab869-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.291130 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.290773 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/59c7afc9-5012-4dae-8cea-2575633ab869-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.291130 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.290787 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/59c7afc9-5012-4dae-8cea-2575633ab869-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.291130 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.290809 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/59c7afc9-5012-4dae-8cea-2575633ab869-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.291130 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.290836 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59c7afc9-5012-4dae-8cea-2575633ab869-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.291130 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.290865 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/59c7afc9-5012-4dae-8cea-2575633ab869-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.291130 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.290900 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/59c7afc9-5012-4dae-8cea-2575633ab869-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.291130 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.290925 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5gc8\" (UniqueName: \"kubernetes.io/projected/59c7afc9-5012-4dae-8cea-2575633ab869-kube-api-access-z5gc8\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.291130 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.290954 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/59c7afc9-5012-4dae-8cea-2575633ab869-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.291130 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.290977 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59c7afc9-5012-4dae-8cea-2575633ab869-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.291130 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.291039 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/59c7afc9-5012-4dae-8cea-2575633ab869-config\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.291130 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.291084 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59c7afc9-5012-4dae-8cea-2575633ab869-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.291130 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.291117 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/59c7afc9-5012-4dae-8cea-2575633ab869-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.291726 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.291145 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/59c7afc9-5012-4dae-8cea-2575633ab869-config-out\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.291726 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.291170 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/59c7afc9-5012-4dae-8cea-2575633ab869-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.291726 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.291197 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/59c7afc9-5012-4dae-8cea-2575633ab869-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.291726 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.291224 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/59c7afc9-5012-4dae-8cea-2575633ab869-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.291934 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.291792 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/59c7afc9-5012-4dae-8cea-2575633ab869-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.292456 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.292429 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59c7afc9-5012-4dae-8cea-2575633ab869-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.293966 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.293938 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/59c7afc9-5012-4dae-8cea-2575633ab869-web-config\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.294077 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.294064 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/59c7afc9-5012-4dae-8cea-2575633ab869-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.294133 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.294064 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/59c7afc9-5012-4dae-8cea-2575633ab869-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.294669 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.294333 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/59c7afc9-5012-4dae-8cea-2575633ab869-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.294669 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.294645 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/59c7afc9-5012-4dae-8cea-2575633ab869-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.296388 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.294722 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/59c7afc9-5012-4dae-8cea-2575633ab869-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.296388 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.294903 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/59c7afc9-5012-4dae-8cea-2575633ab869-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.296388 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.295068 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59c7afc9-5012-4dae-8cea-2575633ab869-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.296388 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.295400 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/59c7afc9-5012-4dae-8cea-2575633ab869-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.296388 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.295488 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/59c7afc9-5012-4dae-8cea-2575633ab869-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.296388 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.295732 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59c7afc9-5012-4dae-8cea-2575633ab869-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.296388 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.296191 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/59c7afc9-5012-4dae-8cea-2575633ab869-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.296936 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.296912 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/59c7afc9-5012-4dae-8cea-2575633ab869-config\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.297109 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.297087 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/59c7afc9-5012-4dae-8cea-2575633ab869-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.297169 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.297149 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/59c7afc9-5012-4dae-8cea-2575633ab869-config-out\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.297264 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.297248 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/59c7afc9-5012-4dae-8cea-2575633ab869-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.302332 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.302283 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5gc8\" (UniqueName: \"kubernetes.io/projected/59c7afc9-5012-4dae-8cea-2575633ab869-kube-api-access-z5gc8\") pod \"prometheus-k8s-0\" (UID: \"59c7afc9-5012-4dae-8cea-2575633ab869\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.400630 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.400588 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:24.685667 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:24.685647 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 03:54:24.688148 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:54:24.688117 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59c7afc9_5012_4dae_8cea_2575633ab869.slice/crio-14f1bf4777d6252d23363a357a712d8ee39e24c24728c0a326b11f669abcc178 WatchSource:0}: Error finding container 14f1bf4777d6252d23363a357a712d8ee39e24c24728c0a326b11f669abcc178: Status 404 returned error can't find the container with id 14f1bf4777d6252d23363a357a712d8ee39e24c24728c0a326b11f669abcc178 Sep 10 03:54:25.002738 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:25.002639 2562 generic.go:356] "Generic (PLEG): container finished" podID="59c7afc9-5012-4dae-8cea-2575633ab869" containerID="2fd4412457ff4bb97d302904ee37eb57e5f3095dcbed5d4e2e2b6e4232b025c1" exitCode=0 Sep 10 03:54:25.002738 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:25.002691 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"59c7afc9-5012-4dae-8cea-2575633ab869","Type":"ContainerDied","Data":"2fd4412457ff4bb97d302904ee37eb57e5f3095dcbed5d4e2e2b6e4232b025c1"} Sep 10 03:54:25.003170 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:25.002757 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"59c7afc9-5012-4dae-8cea-2575633ab869","Type":"ContainerStarted","Data":"14f1bf4777d6252d23363a357a712d8ee39e24c24728c0a326b11f669abcc178"} Sep 10 03:54:25.604384 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:25.604279 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfe42449-c4ae-4b63-be11-9f75095ed0a9" path="/var/lib/kubelet/pods/cfe42449-c4ae-4b63-be11-9f75095ed0a9/volumes" Sep 10 03:54:26.011012 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:26.010972 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"59c7afc9-5012-4dae-8cea-2575633ab869","Type":"ContainerStarted","Data":"ac4a118ffa5765d1a33d94168d9d7ed2fd4be504f3b32ce334d7796640fead5f"} Sep 10 03:54:26.011497 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:26.011066 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"59c7afc9-5012-4dae-8cea-2575633ab869","Type":"ContainerStarted","Data":"5cf3e2d91cb2a1b6396bc8314fca0fcb9fefa662879d781b8e133ab6c3e4a14a"} Sep 10 03:54:26.011497 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:26.011097 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"59c7afc9-5012-4dae-8cea-2575633ab869","Type":"ContainerStarted","Data":"088d0021d8fe44a5328dcff40c47ea89cb6e7db1c1e340eb1da32c8b3aafe56b"} Sep 10 03:54:26.011497 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:26.011123 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"59c7afc9-5012-4dae-8cea-2575633ab869","Type":"ContainerStarted","Data":"998edc3f3a9d5245d29fa04a0001725cb38083882ed1a047fd167113fbe8c4c0"} Sep 10 03:54:26.011497 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:26.011146 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"59c7afc9-5012-4dae-8cea-2575633ab869","Type":"ContainerStarted","Data":"2542c002eaeac2f65eae12aa4f5226a15397c55b3dee99009203a39cdcafe43c"} Sep 10 03:54:26.011497 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:26.011169 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"59c7afc9-5012-4dae-8cea-2575633ab869","Type":"ContainerStarted","Data":"3b8214234dfb7e43b803c8313bdb71f6041db31cd21957f39a79308495adc225"} Sep 10 03:54:26.036567 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:26.036494 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.016970577 podStartE2EDuration="2.036468867s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.49829ms" isStatefulPod=false podCreationTimestamp="2026-09-10 03:54:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:54:26.03548814 +0000 UTC m=+392.911551294" watchObservedRunningTime="2026-09-10 03:54:26.036468867 +0000 UTC m=+392.912532020" Sep 10 03:54:26.321566 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:26.321476 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:54:26.374817 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:26.374780 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:54:29.401107 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:29.401061 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:54:30.838081 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:30.838044 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:54:30.899498 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:30.899458 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:54:30.977141 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:30.977106 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:54:31.574387 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:31.574348 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:54:31.676011 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:31.675974 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:54:41.235554 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:41.235515 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:54:41.314178 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:41.314144 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:54:41.368199 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:41.368160 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:54:41.466327 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:41.466274 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:54:41.519827 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:54:41.519735 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:55:15.426012 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:55:15.425968 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:55:15.475759 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:55:15.475725 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:55:24.401810 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:55:24.401765 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:55:24.417840 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:55:24.417806 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:55:25.267809 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:55:25.267780 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 03:55:27.630572 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:55:27.630529 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:55:27.732449 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:55:27.732403 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:55:54.285624 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:55:54.285581 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:55:55.061178 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:55:55.061134 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:55:57.888675 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:55:57.888641 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:55:57.940150 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:55:57.940117 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:55:59.395100 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:55:59.395075 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:55:59.513562 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:55:59.513529 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:55:59.621273 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:55:59.621239 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:55:59.727529 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:55:59.727447 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:56:01.794460 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:01.794424 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:56:02.009606 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:02.009567 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:56:04.684808 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:04.684768 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:56:04.739880 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:04.739846 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:56:06.896386 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:06.896350 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:56:06.995091 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:06.995050 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:56:09.411948 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:09.411902 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:56:10.691081 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:10.691052 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:56:10.791852 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:10.791824 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:56:10.964961 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:10.964870 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:56:16.939145 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:16.939110 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:56:17.040496 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:17.040458 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:56:18.870782 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:18.870740 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:56:25.978985 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:25.978952 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:56:32.905218 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:32.905178 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:56:33.005974 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:33.005939 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:56:33.568661 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:33.568623 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:56:33.788915 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:33.788871 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:56:33.889059 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:33.889031 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:56:33.928532 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:33.928494 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-797557847b-nb65p"] Sep 10 03:56:33.932049 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:33.932023 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-797557847b-nb65p" Sep 10 03:56:33.937206 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:33.937172 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 10 03:56:33.937439 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:33.937416 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 10 03:56:33.937530 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:33.937512 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 10 03:56:33.937699 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:33.937673 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 10 03:56:33.940329 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:33.940308 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-jgxxc\"" Sep 10 03:56:33.955008 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:33.954975 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-797557847b-nb65p"] Sep 10 03:56:34.067566 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:34.067534 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:56:34.082717 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:34.082680 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a0f70b9-e517-4704-8019-12e9cfd9fd80-tmp\") pod \"opendatahub-operator-controller-manager-797557847b-nb65p\" (UID: \"6a0f70b9-e517-4704-8019-12e9cfd9fd80\") " pod="opendatahub/opendatahub-operator-controller-manager-797557847b-nb65p" Sep 10 03:56:34.082875 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:34.082733 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gcfg\" (UniqueName: \"kubernetes.io/projected/6a0f70b9-e517-4704-8019-12e9cfd9fd80-kube-api-access-5gcfg\") pod \"opendatahub-operator-controller-manager-797557847b-nb65p\" (UID: \"6a0f70b9-e517-4704-8019-12e9cfd9fd80\") " pod="opendatahub/opendatahub-operator-controller-manager-797557847b-nb65p" Sep 10 03:56:34.082875 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:34.082759 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6a0f70b9-e517-4704-8019-12e9cfd9fd80-apiservice-cert\") pod \"opendatahub-operator-controller-manager-797557847b-nb65p\" (UID: \"6a0f70b9-e517-4704-8019-12e9cfd9fd80\") " pod="opendatahub/opendatahub-operator-controller-manager-797557847b-nb65p" Sep 10 03:56:34.082875 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:34.082799 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6a0f70b9-e517-4704-8019-12e9cfd9fd80-manifests\") pod \"opendatahub-operator-controller-manager-797557847b-nb65p\" (UID: \"6a0f70b9-e517-4704-8019-12e9cfd9fd80\") " pod="opendatahub/opendatahub-operator-controller-manager-797557847b-nb65p" Sep 10 03:56:34.082875 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:34.082816 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6a0f70b9-e517-4704-8019-12e9cfd9fd80-webhook-cert\") pod \"opendatahub-operator-controller-manager-797557847b-nb65p\" (UID: \"6a0f70b9-e517-4704-8019-12e9cfd9fd80\") " pod="opendatahub/opendatahub-operator-controller-manager-797557847b-nb65p" Sep 10 03:56:34.167523 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:34.167440 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:56:34.183656 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:34.183620 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a0f70b9-e517-4704-8019-12e9cfd9fd80-tmp\") pod \"opendatahub-operator-controller-manager-797557847b-nb65p\" (UID: \"6a0f70b9-e517-4704-8019-12e9cfd9fd80\") " pod="opendatahub/opendatahub-operator-controller-manager-797557847b-nb65p" Sep 10 03:56:34.183757 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:34.183673 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gcfg\" (UniqueName: \"kubernetes.io/projected/6a0f70b9-e517-4704-8019-12e9cfd9fd80-kube-api-access-5gcfg\") pod \"opendatahub-operator-controller-manager-797557847b-nb65p\" (UID: \"6a0f70b9-e517-4704-8019-12e9cfd9fd80\") " pod="opendatahub/opendatahub-operator-controller-manager-797557847b-nb65p" Sep 10 03:56:34.183757 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:34.183698 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6a0f70b9-e517-4704-8019-12e9cfd9fd80-apiservice-cert\") pod \"opendatahub-operator-controller-manager-797557847b-nb65p\" (UID: \"6a0f70b9-e517-4704-8019-12e9cfd9fd80\") " pod="opendatahub/opendatahub-operator-controller-manager-797557847b-nb65p" Sep 10 03:56:34.183757 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:34.183740 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6a0f70b9-e517-4704-8019-12e9cfd9fd80-manifests\") pod \"opendatahub-operator-controller-manager-797557847b-nb65p\" (UID: \"6a0f70b9-e517-4704-8019-12e9cfd9fd80\") " pod="opendatahub/opendatahub-operator-controller-manager-797557847b-nb65p" Sep 10 03:56:34.183896 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:34.183758 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6a0f70b9-e517-4704-8019-12e9cfd9fd80-webhook-cert\") pod \"opendatahub-operator-controller-manager-797557847b-nb65p\" (UID: \"6a0f70b9-e517-4704-8019-12e9cfd9fd80\") " pod="opendatahub/opendatahub-operator-controller-manager-797557847b-nb65p" Sep 10 03:56:34.184168 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:34.184143 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6a0f70b9-e517-4704-8019-12e9cfd9fd80-manifests\") pod \"opendatahub-operator-controller-manager-797557847b-nb65p\" (UID: \"6a0f70b9-e517-4704-8019-12e9cfd9fd80\") " pod="opendatahub/opendatahub-operator-controller-manager-797557847b-nb65p" Sep 10 03:56:34.186058 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:34.186031 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a0f70b9-e517-4704-8019-12e9cfd9fd80-tmp\") pod \"opendatahub-operator-controller-manager-797557847b-nb65p\" (UID: \"6a0f70b9-e517-4704-8019-12e9cfd9fd80\") " pod="opendatahub/opendatahub-operator-controller-manager-797557847b-nb65p" Sep 10 03:56:34.186192 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:34.186176 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6a0f70b9-e517-4704-8019-12e9cfd9fd80-apiservice-cert\") pod \"opendatahub-operator-controller-manager-797557847b-nb65p\" (UID: \"6a0f70b9-e517-4704-8019-12e9cfd9fd80\") " pod="opendatahub/opendatahub-operator-controller-manager-797557847b-nb65p" Sep 10 03:56:34.186233 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:34.186206 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6a0f70b9-e517-4704-8019-12e9cfd9fd80-webhook-cert\") pod \"opendatahub-operator-controller-manager-797557847b-nb65p\" (UID: \"6a0f70b9-e517-4704-8019-12e9cfd9fd80\") " pod="opendatahub/opendatahub-operator-controller-manager-797557847b-nb65p" Sep 10 03:56:34.191815 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:34.191792 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gcfg\" (UniqueName: \"kubernetes.io/projected/6a0f70b9-e517-4704-8019-12e9cfd9fd80-kube-api-access-5gcfg\") pod \"opendatahub-operator-controller-manager-797557847b-nb65p\" (UID: \"6a0f70b9-e517-4704-8019-12e9cfd9fd80\") " pod="opendatahub/opendatahub-operator-controller-manager-797557847b-nb65p" Sep 10 03:56:34.242901 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:34.242859 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-797557847b-nb65p" Sep 10 03:56:34.277480 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:34.277442 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:56:34.369045 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:34.369011 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:56:34.468799 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:34.468713 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:56:34.522771 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:34.522745 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-797557847b-nb65p"] Sep 10 03:56:34.525660 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:56:34.525632 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a0f70b9_e517_4704_8019_12e9cfd9fd80.slice/crio-6441fd0b7cd9e8a3057076d70465a6234424f90ad075f836df9c4192772c9dc6 WatchSource:0}: Error finding container 6441fd0b7cd9e8a3057076d70465a6234424f90ad075f836df9c4192772c9dc6: Status 404 returned error can't find the container with id 6441fd0b7cd9e8a3057076d70465a6234424f90ad075f836df9c4192772c9dc6 Sep 10 03:56:34.568815 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:34.568784 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:56:35.526563 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:35.526520 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-797557847b-nb65p" event={"ID":"6a0f70b9-e517-4704-8019-12e9cfd9fd80","Type":"ContainerStarted","Data":"6441fd0b7cd9e8a3057076d70465a6234424f90ad075f836df9c4192772c9dc6"} Sep 10 03:56:38.541603 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:38.541565 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-797557847b-nb65p" event={"ID":"6a0f70b9-e517-4704-8019-12e9cfd9fd80","Type":"ContainerStarted","Data":"4d39c8e36e48709be2a8b903ba7f2d98f29449cfe7f616bbcce3d14e94ac931a"} Sep 10 03:56:39.083811 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:39.083757 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:56:39.184352 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:39.184315 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:56:39.548109 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:39.548069 2562 generic.go:356] "Generic (PLEG): container finished" podID="6a0f70b9-e517-4704-8019-12e9cfd9fd80" containerID="4d39c8e36e48709be2a8b903ba7f2d98f29449cfe7f616bbcce3d14e94ac931a" exitCode=0 Sep 10 03:56:39.548524 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:39.548150 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-797557847b-nb65p" event={"ID":"6a0f70b9-e517-4704-8019-12e9cfd9fd80","Type":"ContainerDied","Data":"4d39c8e36e48709be2a8b903ba7f2d98f29449cfe7f616bbcce3d14e94ac931a"} Sep 10 03:56:40.554038 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:40.554006 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-797557847b-nb65p" event={"ID":"6a0f70b9-e517-4704-8019-12e9cfd9fd80","Type":"ContainerStarted","Data":"44b88e1d7d8223b5b2c2617ce2b5242285cb38c47171a457e1b217f6da1356f1"} Sep 10 03:56:40.554434 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:40.554123 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-797557847b-nb65p" Sep 10 03:56:40.575051 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:40.574993 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-797557847b-nb65p" podStartSLOduration=3.175560208 podStartE2EDuration="7.574974455s" totalImagesPullingTime="3.328875483s" totalInitContainerRuntime="1.070538764s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:56:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:56:40.572854204 +0000 UTC m=+527.448917356" watchObservedRunningTime="2026-09-10 03:56:40.574974455 +0000 UTC m=+527.451037606" Sep 10 03:56:47.251465 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:47.251378 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:56:51.531976 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:51.531939 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-78c97bdccc-lrj57"] Sep 10 03:56:51.537564 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:51.537538 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-78c97bdccc-lrj57" Sep 10 03:56:51.539979 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:51.539950 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Sep 10 03:56:51.540883 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:51.540859 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Sep 10 03:56:51.541068 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:51.541053 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Sep 10 03:56:51.541116 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:51.541091 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Sep 10 03:56:51.541164 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:51.541111 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Sep 10 03:56:51.541214 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:51.541185 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-9rsl5\"" Sep 10 03:56:51.543802 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:51.543774 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-78c97bdccc-lrj57"] Sep 10 03:56:51.560701 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:51.560671 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-797557847b-nb65p" Sep 10 03:56:51.638211 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:51.638173 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/43dacf9e-c15a-46f4-bd1d-1c114eb1a458-manager-config\") pod \"lws-controller-manager-78c97bdccc-lrj57\" (UID: \"43dacf9e-c15a-46f4-bd1d-1c114eb1a458\") " pod="openshift-lws-operator/lws-controller-manager-78c97bdccc-lrj57" Sep 10 03:56:51.638211 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:51.638212 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/43dacf9e-c15a-46f4-bd1d-1c114eb1a458-cert\") pod \"lws-controller-manager-78c97bdccc-lrj57\" (UID: \"43dacf9e-c15a-46f4-bd1d-1c114eb1a458\") " pod="openshift-lws-operator/lws-controller-manager-78c97bdccc-lrj57" Sep 10 03:56:51.638474 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:51.638350 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s958c\" (UniqueName: \"kubernetes.io/projected/43dacf9e-c15a-46f4-bd1d-1c114eb1a458-kube-api-access-s958c\") pod \"lws-controller-manager-78c97bdccc-lrj57\" (UID: \"43dacf9e-c15a-46f4-bd1d-1c114eb1a458\") " pod="openshift-lws-operator/lws-controller-manager-78c97bdccc-lrj57" Sep 10 03:56:51.638474 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:51.638397 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/43dacf9e-c15a-46f4-bd1d-1c114eb1a458-metrics-cert\") pod \"lws-controller-manager-78c97bdccc-lrj57\" (UID: \"43dacf9e-c15a-46f4-bd1d-1c114eb1a458\") " pod="openshift-lws-operator/lws-controller-manager-78c97bdccc-lrj57" Sep 10 03:56:51.739768 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:51.739724 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s958c\" (UniqueName: \"kubernetes.io/projected/43dacf9e-c15a-46f4-bd1d-1c114eb1a458-kube-api-access-s958c\") pod \"lws-controller-manager-78c97bdccc-lrj57\" (UID: \"43dacf9e-c15a-46f4-bd1d-1c114eb1a458\") " pod="openshift-lws-operator/lws-controller-manager-78c97bdccc-lrj57" Sep 10 03:56:51.739768 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:51.739771 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/43dacf9e-c15a-46f4-bd1d-1c114eb1a458-metrics-cert\") pod \"lws-controller-manager-78c97bdccc-lrj57\" (UID: \"43dacf9e-c15a-46f4-bd1d-1c114eb1a458\") " pod="openshift-lws-operator/lws-controller-manager-78c97bdccc-lrj57" Sep 10 03:56:51.740021 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:51.739862 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/43dacf9e-c15a-46f4-bd1d-1c114eb1a458-manager-config\") pod \"lws-controller-manager-78c97bdccc-lrj57\" (UID: \"43dacf9e-c15a-46f4-bd1d-1c114eb1a458\") " pod="openshift-lws-operator/lws-controller-manager-78c97bdccc-lrj57" Sep 10 03:56:51.740021 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:51.739889 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/43dacf9e-c15a-46f4-bd1d-1c114eb1a458-cert\") pod \"lws-controller-manager-78c97bdccc-lrj57\" (UID: \"43dacf9e-c15a-46f4-bd1d-1c114eb1a458\") " pod="openshift-lws-operator/lws-controller-manager-78c97bdccc-lrj57" Sep 10 03:56:51.740601 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:51.740579 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/43dacf9e-c15a-46f4-bd1d-1c114eb1a458-manager-config\") pod \"lws-controller-manager-78c97bdccc-lrj57\" (UID: \"43dacf9e-c15a-46f4-bd1d-1c114eb1a458\") " pod="openshift-lws-operator/lws-controller-manager-78c97bdccc-lrj57" Sep 10 03:56:51.742330 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:51.742287 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/43dacf9e-c15a-46f4-bd1d-1c114eb1a458-metrics-cert\") pod \"lws-controller-manager-78c97bdccc-lrj57\" (UID: \"43dacf9e-c15a-46f4-bd1d-1c114eb1a458\") " pod="openshift-lws-operator/lws-controller-manager-78c97bdccc-lrj57" Sep 10 03:56:51.742412 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:51.742363 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/43dacf9e-c15a-46f4-bd1d-1c114eb1a458-cert\") pod \"lws-controller-manager-78c97bdccc-lrj57\" (UID: \"43dacf9e-c15a-46f4-bd1d-1c114eb1a458\") " pod="openshift-lws-operator/lws-controller-manager-78c97bdccc-lrj57" Sep 10 03:56:51.748004 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:51.747984 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s958c\" (UniqueName: \"kubernetes.io/projected/43dacf9e-c15a-46f4-bd1d-1c114eb1a458-kube-api-access-s958c\") pod \"lws-controller-manager-78c97bdccc-lrj57\" (UID: \"43dacf9e-c15a-46f4-bd1d-1c114eb1a458\") " pod="openshift-lws-operator/lws-controller-manager-78c97bdccc-lrj57" Sep 10 03:56:51.848241 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:51.848143 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-78c97bdccc-lrj57" Sep 10 03:56:52.118432 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:52.118354 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-78c97bdccc-lrj57"] Sep 10 03:56:52.121597 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:56:52.121557 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43dacf9e_c15a_46f4_bd1d_1c114eb1a458.slice/crio-6cc6354cec79b234d06093810ef10ff1e31736fbcf64fcea0d73d3590fcbf340 WatchSource:0}: Error finding container 6cc6354cec79b234d06093810ef10ff1e31736fbcf64fcea0d73d3590fcbf340: Status 404 returned error can't find the container with id 6cc6354cec79b234d06093810ef10ff1e31736fbcf64fcea0d73d3590fcbf340 Sep 10 03:56:52.612016 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:52.611975 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-78c97bdccc-lrj57" event={"ID":"43dacf9e-c15a-46f4-bd1d-1c114eb1a458","Type":"ContainerStarted","Data":"6cc6354cec79b234d06093810ef10ff1e31736fbcf64fcea0d73d3590fcbf340"} Sep 10 03:56:53.260794 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:53.260723 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:56:53.368742 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:53.368703 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:56:53.731202 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:53.731163 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:56:53.759872 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:53.759703 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5dcb5c9454-wstk2"] Sep 10 03:56:53.765238 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:53.765200 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5dcb5c9454-wstk2" Sep 10 03:56:53.767933 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:53.767907 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Sep 10 03:56:53.768069 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:53.767975 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-p9sq7\"" Sep 10 03:56:53.768837 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:53.768815 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 10 03:56:53.768837 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:53.768834 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Sep 10 03:56:53.769036 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:53.768870 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 10 03:56:53.780206 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:53.780180 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5dcb5c9454-wstk2"] Sep 10 03:56:53.825557 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:53.825521 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:56:53.960165 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:53.960132 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsw79\" (UniqueName: \"kubernetes.io/projected/41b98efd-b6d8-413c-9017-10e71dabaacf-kube-api-access-lsw79\") pod \"kube-auth-proxy-5dcb5c9454-wstk2\" (UID: \"41b98efd-b6d8-413c-9017-10e71dabaacf\") " pod="openshift-ingress/kube-auth-proxy-5dcb5c9454-wstk2" Sep 10 03:56:53.960350 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:53.960186 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41b98efd-b6d8-413c-9017-10e71dabaacf-tls-certs\") pod \"kube-auth-proxy-5dcb5c9454-wstk2\" (UID: \"41b98efd-b6d8-413c-9017-10e71dabaacf\") " pod="openshift-ingress/kube-auth-proxy-5dcb5c9454-wstk2" Sep 10 03:56:53.960350 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:53.960262 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41b98efd-b6d8-413c-9017-10e71dabaacf-tmp\") pod \"kube-auth-proxy-5dcb5c9454-wstk2\" (UID: \"41b98efd-b6d8-413c-9017-10e71dabaacf\") " pod="openshift-ingress/kube-auth-proxy-5dcb5c9454-wstk2" Sep 10 03:56:54.061787 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:54.061706 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsw79\" (UniqueName: \"kubernetes.io/projected/41b98efd-b6d8-413c-9017-10e71dabaacf-kube-api-access-lsw79\") pod \"kube-auth-proxy-5dcb5c9454-wstk2\" (UID: \"41b98efd-b6d8-413c-9017-10e71dabaacf\") " pod="openshift-ingress/kube-auth-proxy-5dcb5c9454-wstk2" Sep 10 03:56:54.061787 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:54.061756 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41b98efd-b6d8-413c-9017-10e71dabaacf-tls-certs\") pod \"kube-auth-proxy-5dcb5c9454-wstk2\" (UID: \"41b98efd-b6d8-413c-9017-10e71dabaacf\") " pod="openshift-ingress/kube-auth-proxy-5dcb5c9454-wstk2" Sep 10 03:56:54.062022 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:54.061887 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41b98efd-b6d8-413c-9017-10e71dabaacf-tmp\") pod \"kube-auth-proxy-5dcb5c9454-wstk2\" (UID: \"41b98efd-b6d8-413c-9017-10e71dabaacf\") " pod="openshift-ingress/kube-auth-proxy-5dcb5c9454-wstk2" Sep 10 03:56:54.064119 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:54.064098 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41b98efd-b6d8-413c-9017-10e71dabaacf-tmp\") pod \"kube-auth-proxy-5dcb5c9454-wstk2\" (UID: \"41b98efd-b6d8-413c-9017-10e71dabaacf\") " pod="openshift-ingress/kube-auth-proxy-5dcb5c9454-wstk2" Sep 10 03:56:54.064461 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:54.064440 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41b98efd-b6d8-413c-9017-10e71dabaacf-tls-certs\") pod \"kube-auth-proxy-5dcb5c9454-wstk2\" (UID: \"41b98efd-b6d8-413c-9017-10e71dabaacf\") " pod="openshift-ingress/kube-auth-proxy-5dcb5c9454-wstk2" Sep 10 03:56:54.069535 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:54.069515 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsw79\" (UniqueName: \"kubernetes.io/projected/41b98efd-b6d8-413c-9017-10e71dabaacf-kube-api-access-lsw79\") pod \"kube-auth-proxy-5dcb5c9454-wstk2\" (UID: \"41b98efd-b6d8-413c-9017-10e71dabaacf\") " pod="openshift-ingress/kube-auth-proxy-5dcb5c9454-wstk2" Sep 10 03:56:54.078327 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:54.078278 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5dcb5c9454-wstk2" Sep 10 03:56:54.872056 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:54.871835 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:56:54.971464 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:54.971198 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:56:55.181230 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:55.181198 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5dcb5c9454-wstk2"] Sep 10 03:56:55.183898 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:56:55.183871 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41b98efd_b6d8_413c_9017_10e71dabaacf.slice/crio-fd570ae9234163538af381ba7e5e1733abd75ca5e4da5860fc82a658a4f7e1e5 WatchSource:0}: Error finding container fd570ae9234163538af381ba7e5e1733abd75ca5e4da5860fc82a658a4f7e1e5: Status 404 returned error can't find the container with id fd570ae9234163538af381ba7e5e1733abd75ca5e4da5860fc82a658a4f7e1e5 Sep 10 03:56:55.630490 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:55.630398 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5dcb5c9454-wstk2" event={"ID":"41b98efd-b6d8-413c-9017-10e71dabaacf","Type":"ContainerStarted","Data":"fd570ae9234163538af381ba7e5e1733abd75ca5e4da5860fc82a658a4f7e1e5"} Sep 10 03:56:55.632160 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:55.632126 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-78c97bdccc-lrj57" event={"ID":"43dacf9e-c15a-46f4-bd1d-1c114eb1a458","Type":"ContainerStarted","Data":"7dba0848475dac4bdf64e923478ce557170d756fb0313644879e14e3d4b60119"} Sep 10 03:56:55.632327 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:55.632275 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-78c97bdccc-lrj57" Sep 10 03:56:55.648118 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:55.648056 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-78c97bdccc-lrj57" podStartSLOduration=1.7886691620000001 podStartE2EDuration="4.648034891s" totalImagesPullingTime="2.859365729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:56:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:56:55.646513824 +0000 UTC m=+542.522576977" watchObservedRunningTime="2026-09-10 03:56:55.648034891 +0000 UTC m=+542.524098044" Sep 10 03:56:55.783072 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:55.783033 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:56:58.647609 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:58.647568 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5dcb5c9454-wstk2" event={"ID":"41b98efd-b6d8-413c-9017-10e71dabaacf","Type":"ContainerStarted","Data":"2cbaed36c3cc992e967cbd9215a4a6bd7f920541b2df42ed382be597bc9df919"} Sep 10 03:56:58.664276 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:58.664217 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5dcb5c9454-wstk2" podStartSLOduration=3.168163114 podStartE2EDuration="5.664199785s" totalImagesPullingTime="2.496036671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:56:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:56:58.662677854 +0000 UTC m=+545.538741009" watchObservedRunningTime="2026-09-10 03:56:58.664199785 +0000 UTC m=+545.540262936" Sep 10 03:56:59.727387 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:59.727355 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:56:59.825882 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:56:59.825839 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:57:00.263596 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:57:00.263560 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:57:00.448074 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:57:00.448040 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:57:00.648841 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:57:00.648808 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:57:04.287890 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:57:04.287856 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:57:04.389477 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:57:04.389444 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:57:04.490447 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:57:04.490388 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:57:04.690419 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:57:04.690384 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:57:04.765557 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:57:04.765517 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:57:04.834792 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:57:04.834754 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:57:06.639937 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:57:06.639901 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-78c97bdccc-lrj57" Sep 10 03:57:13.425656 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:57:13.425616 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:57:13.536120 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:57:13.536084 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:57:13.782555 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:57:13.782462 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:57:13.879634 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:57:13.879597 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:57:14.713163 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:57:14.713121 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:57:14.823473 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:57:14.823437 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:57:14.921853 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:57:14.921821 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:57:49.267969 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:57:49.267933 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:57:49.368662 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:57:49.368619 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:57:53.543588 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:57:53.543558 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hf7wl_50451d7c-07fb-4641-a36f-d142e72b5e81/console-operator/2.log" Sep 10 03:57:53.544552 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:57:53.544515 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hf7wl_50451d7c-07fb-4641-a36f-d142e72b5e81/console-operator/2.log" Sep 10 03:57:53.554499 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:57:53.554468 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fl7hj_8459a996-8d64-4c23-9e92-1a5bcca253b5/ovn-acl-logging/0.log" Sep 10 03:57:53.555333 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:57:53.555308 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fl7hj_8459a996-8d64-4c23-9e92-1a5bcca253b5/ovn-acl-logging/0.log" Sep 10 03:57:57.511828 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:57:57.511783 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:57:58.607138 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:57:58.607100 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:58:05.343025 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:05.342969 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:58:05.445384 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:05.445344 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:58:19.449950 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:19.449911 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:58:19.506852 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:19.506817 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:58:26.065522 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:26.065482 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:58:28.454744 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:28.454705 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:58:28.854589 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:28.854489 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:58:30.255929 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:30.255891 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:58:30.356532 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:30.356494 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:58:30.460308 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:30.460273 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:58:30.570829 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:30.570733 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:58:30.781229 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:30.781196 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:58:31.244878 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:31.244846 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:58:31.643422 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:31.643390 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:58:32.047734 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:32.047648 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:58:32.519723 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:32.519686 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:58:33.522821 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:33.522788 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:58:35.687063 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:35.687030 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:58:35.785173 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:35.785131 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:58:38.662473 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:38.662437 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:58:38.763975 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:38.763940 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:58:42.467769 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:42.467736 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:58:42.525309 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:42.525265 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-vvdhn"] Sep 10 03:58:42.528536 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:42.528515 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vvdhn" Sep 10 03:58:42.530917 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:42.530894 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 10 03:58:42.531865 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:42.531842 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 10 03:58:42.532750 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:42.532728 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Sep 10 03:58:42.532871 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:42.532806 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Sep 10 03:58:42.533033 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:42.533012 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-n7cbv\"" Sep 10 03:58:42.537242 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:42.537215 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-vvdhn"] Sep 10 03:58:42.541671 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:42.541644 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:58:42.648515 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:42.648477 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:58:42.702012 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:42.701970 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/670d97b5-b0f6-4141-a2cf-955aa87e779b-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-vvdhn\" (UID: \"670d97b5-b0f6-4141-a2cf-955aa87e779b\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vvdhn" Sep 10 03:58:42.702012 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:42.702008 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/670d97b5-b0f6-4141-a2cf-955aa87e779b-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-vvdhn\" (UID: \"670d97b5-b0f6-4141-a2cf-955aa87e779b\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vvdhn" Sep 10 03:58:42.702236 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:42.702049 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2g7tx\" (UniqueName: \"kubernetes.io/projected/670d97b5-b0f6-4141-a2cf-955aa87e779b-kube-api-access-2g7tx\") pod \"kuadrant-console-plugin-fbdd4f445-vvdhn\" (UID: \"670d97b5-b0f6-4141-a2cf-955aa87e779b\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vvdhn" Sep 10 03:58:42.748884 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:42.748791 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:58:42.803072 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:42.803035 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/670d97b5-b0f6-4141-a2cf-955aa87e779b-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-vvdhn\" (UID: \"670d97b5-b0f6-4141-a2cf-955aa87e779b\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vvdhn" Sep 10 03:58:42.803268 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:42.803079 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/670d97b5-b0f6-4141-a2cf-955aa87e779b-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-vvdhn\" (UID: \"670d97b5-b0f6-4141-a2cf-955aa87e779b\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vvdhn" Sep 10 03:58:42.803268 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:42.803120 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2g7tx\" (UniqueName: \"kubernetes.io/projected/670d97b5-b0f6-4141-a2cf-955aa87e779b-kube-api-access-2g7tx\") pod \"kuadrant-console-plugin-fbdd4f445-vvdhn\" (UID: \"670d97b5-b0f6-4141-a2cf-955aa87e779b\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vvdhn" Sep 10 03:58:42.803818 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:42.803792 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/670d97b5-b0f6-4141-a2cf-955aa87e779b-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-vvdhn\" (UID: \"670d97b5-b0f6-4141-a2cf-955aa87e779b\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vvdhn" Sep 10 03:58:42.805626 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:42.805601 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/670d97b5-b0f6-4141-a2cf-955aa87e779b-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-vvdhn\" (UID: \"670d97b5-b0f6-4141-a2cf-955aa87e779b\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vvdhn" Sep 10 03:58:42.814495 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:42.814466 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2g7tx\" (UniqueName: \"kubernetes.io/projected/670d97b5-b0f6-4141-a2cf-955aa87e779b-kube-api-access-2g7tx\") pod \"kuadrant-console-plugin-fbdd4f445-vvdhn\" (UID: \"670d97b5-b0f6-4141-a2cf-955aa87e779b\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vvdhn" Sep 10 03:58:42.849489 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:42.849451 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:58:42.849667 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:42.849504 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vvdhn" Sep 10 03:58:43.118947 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:43.118914 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-vvdhn"] Sep 10 03:58:43.121870 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:58:43.121835 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod670d97b5_b0f6_4141_a2cf_955aa87e779b.slice/crio-6589509ba9b558ec3300f692c321a75d9f34632e49287d35643ac90f4c25b907 WatchSource:0}: Error finding container 6589509ba9b558ec3300f692c321a75d9f34632e49287d35643ac90f4c25b907: Status 404 returned error can't find the container with id 6589509ba9b558ec3300f692c321a75d9f34632e49287d35643ac90f4c25b907 Sep 10 03:58:43.123322 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:43.123278 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 03:58:44.113952 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:44.113915 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vvdhn" event={"ID":"670d97b5-b0f6-4141-a2cf-955aa87e779b","Type":"ContainerStarted","Data":"6589509ba9b558ec3300f692c321a75d9f34632e49287d35643ac90f4c25b907"} Sep 10 03:58:47.599134 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:47.599098 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:58:47.698744 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:47.698709 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:58:47.798871 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:47.798838 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:58:48.142104 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:48.142060 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vvdhn" event={"ID":"670d97b5-b0f6-4141-a2cf-955aa87e779b","Type":"ContainerStarted","Data":"a09d9923c9de4438dbe93189b7710883255a3122051ba74f523b156214bc890b"} Sep 10 03:58:48.159852 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:48.159717 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vvdhn" podStartSLOduration=1.385203154 podStartE2EDuration="6.159696948s" totalImagesPullingTime="4.774493794s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:58:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:58:48.158326183 +0000 UTC m=+655.034389337" watchObservedRunningTime="2026-09-10 03:58:48.159696948 +0000 UTC m=+655.035760099" Sep 10 03:58:53.268128 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:53.268086 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tcm5n"] Sep 10 03:58:53.271608 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:53.271584 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tcm5n" Sep 10 03:58:53.273842 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:53.273813 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-425jf\"" Sep 10 03:58:53.287454 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:53.287425 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tcm5n"] Sep 10 03:58:53.301705 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:53.301666 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3540a9af-54f5-4b97-98bc-67882f1235a1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-tcm5n\" (UID: \"3540a9af-54f5-4b97-98bc-67882f1235a1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tcm5n" Sep 10 03:58:53.301897 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:53.301736 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grzfn\" (UniqueName: \"kubernetes.io/projected/3540a9af-54f5-4b97-98bc-67882f1235a1-kube-api-access-grzfn\") pod \"kuadrant-operator-controller-manager-76f8f686bf-tcm5n\" (UID: \"3540a9af-54f5-4b97-98bc-67882f1235a1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tcm5n" Sep 10 03:58:53.402874 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:53.402827 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3540a9af-54f5-4b97-98bc-67882f1235a1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-tcm5n\" (UID: \"3540a9af-54f5-4b97-98bc-67882f1235a1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tcm5n" Sep 10 03:58:53.403076 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:53.402896 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grzfn\" (UniqueName: \"kubernetes.io/projected/3540a9af-54f5-4b97-98bc-67882f1235a1-kube-api-access-grzfn\") pod \"kuadrant-operator-controller-manager-76f8f686bf-tcm5n\" (UID: \"3540a9af-54f5-4b97-98bc-67882f1235a1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tcm5n" Sep 10 03:58:53.403231 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:53.403208 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3540a9af-54f5-4b97-98bc-67882f1235a1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-tcm5n\" (UID: \"3540a9af-54f5-4b97-98bc-67882f1235a1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tcm5n" Sep 10 03:58:53.415262 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:53.415225 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grzfn\" (UniqueName: \"kubernetes.io/projected/3540a9af-54f5-4b97-98bc-67882f1235a1-kube-api-access-grzfn\") pod \"kuadrant-operator-controller-manager-76f8f686bf-tcm5n\" (UID: \"3540a9af-54f5-4b97-98bc-67882f1235a1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tcm5n" Sep 10 03:58:53.587141 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:53.587054 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-425jf\"" Sep 10 03:58:53.595374 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:53.595346 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tcm5n" Sep 10 03:58:53.878054 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:53.877965 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tcm5n"] Sep 10 03:58:53.881258 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:58:53.881226 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3540a9af_54f5_4b97_98bc_67882f1235a1.slice/crio-43c5afe2f2b881004bab9114f10294f014e4269c77faf72d51a89e9d88208d1e WatchSource:0}: Error finding container 43c5afe2f2b881004bab9114f10294f014e4269c77faf72d51a89e9d88208d1e: Status 404 returned error can't find the container with id 43c5afe2f2b881004bab9114f10294f014e4269c77faf72d51a89e9d88208d1e Sep 10 03:58:54.095984 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:54.095949 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:58:54.170761 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:54.170723 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tcm5n" event={"ID":"3540a9af-54f5-4b97-98bc-67882f1235a1","Type":"ContainerStarted","Data":"43c5afe2f2b881004bab9114f10294f014e4269c77faf72d51a89e9d88208d1e"} Sep 10 03:58:54.190936 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:54.190898 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:58:58.192474 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:58.192438 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tcm5n" event={"ID":"3540a9af-54f5-4b97-98bc-67882f1235a1","Type":"ContainerStarted","Data":"0fe29719bba6874e0e1126c8232d9ee209869f29c18fbee926af6e9b6c28f1aa"} Sep 10 03:58:58.192925 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:58.192562 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tcm5n" Sep 10 03:58:58.215173 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:58.215110 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tcm5n" podStartSLOduration=1.578678078 podStartE2EDuration="5.2150916s" totalImagesPullingTime="3.636413522s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:58:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:58:58.213197605 +0000 UTC m=+665.089260759" watchObservedRunningTime="2026-09-10 03:58:58.2150916 +0000 UTC m=+665.091154751" Sep 10 03:58:58.432546 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:58.432509 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:58:58.532912 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:58:58.532790 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:59:02.224799 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:02.224759 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:59:02.284390 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:02.284358 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:59:02.453477 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:02.453444 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:59:02.571886 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:02.571793 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:59:05.312567 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:05.312535 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tcm5n"] Sep 10 03:59:05.313080 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:05.312713 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tcm5n" podUID="3540a9af-54f5-4b97-98bc-67882f1235a1" containerName="manager" containerID="cri-o://0fe29719bba6874e0e1126c8232d9ee209869f29c18fbee926af6e9b6c28f1aa" gracePeriod=10 Sep 10 03:59:05.314652 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:05.314625 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tcm5n" Sep 10 03:59:05.571165 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:05.571090 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tcm5n" Sep 10 03:59:05.715629 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:05.715588 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3540a9af-54f5-4b97-98bc-67882f1235a1-extensions-socket-volume\") pod \"3540a9af-54f5-4b97-98bc-67882f1235a1\" (UID: \"3540a9af-54f5-4b97-98bc-67882f1235a1\") " Sep 10 03:59:05.715629 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:05.715630 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-grzfn\" (UniqueName: \"kubernetes.io/projected/3540a9af-54f5-4b97-98bc-67882f1235a1-kube-api-access-grzfn\") pod \"3540a9af-54f5-4b97-98bc-67882f1235a1\" (UID: \"3540a9af-54f5-4b97-98bc-67882f1235a1\") " Sep 10 03:59:05.716213 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:05.716184 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3540a9af-54f5-4b97-98bc-67882f1235a1-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3540a9af-54f5-4b97-98bc-67882f1235a1" (UID: "3540a9af-54f5-4b97-98bc-67882f1235a1"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 03:59:05.718016 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:05.717996 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3540a9af-54f5-4b97-98bc-67882f1235a1-kube-api-access-grzfn" (OuterVolumeSpecName: "kube-api-access-grzfn") pod "3540a9af-54f5-4b97-98bc-67882f1235a1" (UID: "3540a9af-54f5-4b97-98bc-67882f1235a1"). InnerVolumeSpecName "kube-api-access-grzfn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 03:59:05.816722 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:05.816655 2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3540a9af-54f5-4b97-98bc-67882f1235a1-extensions-socket-volume\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:59:05.816722 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:05.816687 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grzfn\" (UniqueName: \"kubernetes.io/projected/3540a9af-54f5-4b97-98bc-67882f1235a1-kube-api-access-grzfn\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:59:06.229123 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:06.229081 2562 generic.go:356] "Generic (PLEG): container finished" podID="3540a9af-54f5-4b97-98bc-67882f1235a1" containerID="0fe29719bba6874e0e1126c8232d9ee209869f29c18fbee926af6e9b6c28f1aa" exitCode=0 Sep 10 03:59:06.229123 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:06.229121 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tcm5n" Sep 10 03:59:06.229358 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:06.229167 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tcm5n" event={"ID":"3540a9af-54f5-4b97-98bc-67882f1235a1","Type":"ContainerDied","Data":"0fe29719bba6874e0e1126c8232d9ee209869f29c18fbee926af6e9b6c28f1aa"} Sep 10 03:59:06.229358 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:06.229224 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tcm5n" event={"ID":"3540a9af-54f5-4b97-98bc-67882f1235a1","Type":"ContainerDied","Data":"43c5afe2f2b881004bab9114f10294f014e4269c77faf72d51a89e9d88208d1e"} Sep 10 03:59:06.229358 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:06.229247 2562 scope.go:117] "RemoveContainer" containerID="0fe29719bba6874e0e1126c8232d9ee209869f29c18fbee926af6e9b6c28f1aa" Sep 10 03:59:06.237647 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:06.237625 2562 scope.go:117] "RemoveContainer" containerID="0fe29719bba6874e0e1126c8232d9ee209869f29c18fbee926af6e9b6c28f1aa" Sep 10 03:59:06.237953 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:59:06.237933 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fe29719bba6874e0e1126c8232d9ee209869f29c18fbee926af6e9b6c28f1aa\": container with ID starting with 0fe29719bba6874e0e1126c8232d9ee209869f29c18fbee926af6e9b6c28f1aa not found: ID does not exist" containerID="0fe29719bba6874e0e1126c8232d9ee209869f29c18fbee926af6e9b6c28f1aa" Sep 10 03:59:06.238002 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:06.237966 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fe29719bba6874e0e1126c8232d9ee209869f29c18fbee926af6e9b6c28f1aa"} err="failed to get container status \"0fe29719bba6874e0e1126c8232d9ee209869f29c18fbee926af6e9b6c28f1aa\": rpc error: code = NotFound desc = could not find container \"0fe29719bba6874e0e1126c8232d9ee209869f29c18fbee926af6e9b6c28f1aa\": container with ID starting with 0fe29719bba6874e0e1126c8232d9ee209869f29c18fbee926af6e9b6c28f1aa not found: ID does not exist" Sep 10 03:59:06.253221 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:06.253192 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tcm5n"] Sep 10 03:59:06.261901 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:06.261870 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-tcm5n"] Sep 10 03:59:07.604122 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:07.604086 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3540a9af-54f5-4b97-98bc-67882f1235a1" path="/var/lib/kubelet/pods/3540a9af-54f5-4b97-98bc-67882f1235a1/volumes" Sep 10 03:59:07.824102 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:07.824070 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:59:07.930617 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:07.930580 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:59:08.093496 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:08.093464 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:59:08.181885 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:08.181791 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:59:08.279005 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:08.278971 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:59:08.378627 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:08.378593 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:59:10.642235 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:10.642201 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:59:10.739815 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:10.739775 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:59:14.112176 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:14.112144 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:59:14.213977 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:14.213942 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:59:21.629782 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:21.629733 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:59:21.746645 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:21.746600 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:59:33.713153 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:33.713114 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:59:33.826879 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:33.826847 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:59:33.926783 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:33.926742 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:59:34.019802 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:34.019712 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d2rtt"] Sep 10 03:59:34.020107 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:34.020094 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3540a9af-54f5-4b97-98bc-67882f1235a1" containerName="manager" Sep 10 03:59:34.020154 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:34.020109 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3540a9af-54f5-4b97-98bc-67882f1235a1" containerName="manager" Sep 10 03:59:34.020189 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:34.020161 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3540a9af-54f5-4b97-98bc-67882f1235a1" containerName="manager" Sep 10 03:59:34.021891 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:34.021873 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-d2rtt" Sep 10 03:59:34.024078 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:34.024054 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Sep 10 03:59:34.033362 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:34.033337 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d2rtt"] Sep 10 03:59:34.054589 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:34.054545 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chgdw\" (UniqueName: \"kubernetes.io/projected/64f870d7-9485-4051-b64c-ffe5403791c5-kube-api-access-chgdw\") pod \"limitador-limitador-84994f7ffc-d2rtt\" (UID: \"64f870d7-9485-4051-b64c-ffe5403791c5\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-d2rtt" Sep 10 03:59:34.054589 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:34.054596 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/64f870d7-9485-4051-b64c-ffe5403791c5-config-file\") pod \"limitador-limitador-84994f7ffc-d2rtt\" (UID: \"64f870d7-9485-4051-b64c-ffe5403791c5\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-d2rtt" Sep 10 03:59:34.061091 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:34.061057 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d2rtt"] Sep 10 03:59:34.155212 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:34.155170 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/64f870d7-9485-4051-b64c-ffe5403791c5-config-file\") pod \"limitador-limitador-84994f7ffc-d2rtt\" (UID: \"64f870d7-9485-4051-b64c-ffe5403791c5\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-d2rtt" Sep 10 03:59:34.155391 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:34.155323 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chgdw\" (UniqueName: \"kubernetes.io/projected/64f870d7-9485-4051-b64c-ffe5403791c5-kube-api-access-chgdw\") pod \"limitador-limitador-84994f7ffc-d2rtt\" (UID: \"64f870d7-9485-4051-b64c-ffe5403791c5\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-d2rtt" Sep 10 03:59:34.155914 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:34.155895 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/64f870d7-9485-4051-b64c-ffe5403791c5-config-file\") pod \"limitador-limitador-84994f7ffc-d2rtt\" (UID: \"64f870d7-9485-4051-b64c-ffe5403791c5\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-d2rtt" Sep 10 03:59:34.162916 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:34.162891 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chgdw\" (UniqueName: \"kubernetes.io/projected/64f870d7-9485-4051-b64c-ffe5403791c5-kube-api-access-chgdw\") pod \"limitador-limitador-84994f7ffc-d2rtt\" (UID: \"64f870d7-9485-4051-b64c-ffe5403791c5\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-d2rtt" Sep 10 03:59:34.332317 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:34.332182 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-d2rtt" Sep 10 03:59:34.503904 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:34.503862 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-9wlgr"] Sep 10 03:59:34.506355 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:34.506332 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-9wlgr" Sep 10 03:59:34.508491 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:34.508470 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-8xm54\"" Sep 10 03:59:34.514432 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:34.514404 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-9wlgr"] Sep 10 03:59:34.543229 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:34.543197 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:59:34.557495 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:34.557459 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hg87h\" (UniqueName: \"kubernetes.io/projected/0a84a8b2-9ebf-4255-938a-fb5c87022755-kube-api-access-hg87h\") pod \"authorino-7bcfd8f9cd-9wlgr\" (UID: \"0a84a8b2-9ebf-4255-938a-fb5c87022755\") " pod="kuadrant-system/authorino-7bcfd8f9cd-9wlgr" Sep 10 03:59:34.605843 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:34.605817 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d2rtt"] Sep 10 03:59:34.608497 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:59:34.608469 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64f870d7_9485_4051_b64c_ffe5403791c5.slice/crio-9cbdd1a3e537ea8690cdb34e8dcc9c6a5bf4a6c5a71399f6eaad391eb2f52b5d WatchSource:0}: Error finding container 9cbdd1a3e537ea8690cdb34e8dcc9c6a5bf4a6c5a71399f6eaad391eb2f52b5d: Status 404 returned error can't find the container with id 9cbdd1a3e537ea8690cdb34e8dcc9c6a5bf4a6c5a71399f6eaad391eb2f52b5d Sep 10 03:59:34.611854 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:34.611831 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:59:34.658351 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:34.658274 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg87h\" (UniqueName: \"kubernetes.io/projected/0a84a8b2-9ebf-4255-938a-fb5c87022755-kube-api-access-hg87h\") pod \"authorino-7bcfd8f9cd-9wlgr\" (UID: \"0a84a8b2-9ebf-4255-938a-fb5c87022755\") " pod="kuadrant-system/authorino-7bcfd8f9cd-9wlgr" Sep 10 03:59:34.665956 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:34.665928 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hg87h\" (UniqueName: \"kubernetes.io/projected/0a84a8b2-9ebf-4255-938a-fb5c87022755-kube-api-access-hg87h\") pod \"authorino-7bcfd8f9cd-9wlgr\" (UID: \"0a84a8b2-9ebf-4255-938a-fb5c87022755\") " pod="kuadrant-system/authorino-7bcfd8f9cd-9wlgr" Sep 10 03:59:34.817127 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:34.817087 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-9wlgr" Sep 10 03:59:35.086787 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:35.086750 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-9wlgr"] Sep 10 03:59:35.090023 ip-10-0-132-74 kubenswrapper[2562]: W0910 03:59:35.089987 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0a84a8b2_9ebf_4255_938a_fb5c87022755.slice/crio-01e7a1433ea0c2e971d96a954526a817fad2f1e9f57586e2178f9cf3d5d1bb9b WatchSource:0}: Error finding container 01e7a1433ea0c2e971d96a954526a817fad2f1e9f57586e2178f9cf3d5d1bb9b: Status 404 returned error can't find the container with id 01e7a1433ea0c2e971d96a954526a817fad2f1e9f57586e2178f9cf3d5d1bb9b Sep 10 03:59:35.359626 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:35.359579 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-d2rtt" event={"ID":"64f870d7-9485-4051-b64c-ffe5403791c5","Type":"ContainerStarted","Data":"9cbdd1a3e537ea8690cdb34e8dcc9c6a5bf4a6c5a71399f6eaad391eb2f52b5d"} Sep 10 03:59:35.360716 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:35.360682 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-9wlgr" event={"ID":"0a84a8b2-9ebf-4255-938a-fb5c87022755","Type":"ContainerStarted","Data":"01e7a1433ea0c2e971d96a954526a817fad2f1e9f57586e2178f9cf3d5d1bb9b"} Sep 10 03:59:36.367870 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:36.367836 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-d2rtt" event={"ID":"64f870d7-9485-4051-b64c-ffe5403791c5","Type":"ContainerStarted","Data":"76106ebe5359a20496722a90e27c8a5edd3e5d87ecf5ed7c503effcac4edb53a"} Sep 10 03:59:36.368278 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:36.367970 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-d2rtt" Sep 10 03:59:36.385714 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:36.385643 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-d2rtt" podStartSLOduration=1.611868528 podStartE2EDuration="2.385616313s" totalImagesPullingTime="773.747785ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:59:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:59:36.384317802 +0000 UTC m=+703.260380955" watchObservedRunningTime="2026-09-10 03:59:36.385616313 +0000 UTC m=+703.261679467" Sep 10 03:59:37.374422 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:37.374390 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-9wlgr" event={"ID":"0a84a8b2-9ebf-4255-938a-fb5c87022755","Type":"ContainerStarted","Data":"831f9e7a8490905903551c5b1d8176c6efb526e6539c8b147bc5b065a071b185"} Sep 10 03:59:37.389058 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:37.389000 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-9wlgr" podStartSLOduration=1.673495916 podStartE2EDuration="3.388983142s" totalImagesPullingTime="1.715487226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:59:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:59:37.387243478 +0000 UTC m=+704.263306631" watchObservedRunningTime="2026-09-10 03:59:37.388983142 +0000 UTC m=+704.265046293" Sep 10 03:59:38.566520 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:38.566479 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-9wlgr"] Sep 10 03:59:39.391187 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:39.391145 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-9wlgr" podUID="0a84a8b2-9ebf-4255-938a-fb5c87022755" containerName="authorino" containerID="cri-o://831f9e7a8490905903551c5b1d8176c6efb526e6539c8b147bc5b065a071b185" gracePeriod=30 Sep 10 03:59:39.636816 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:39.636786 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-9wlgr" Sep 10 03:59:39.704986 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:39.704890 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hg87h\" (UniqueName: \"kubernetes.io/projected/0a84a8b2-9ebf-4255-938a-fb5c87022755-kube-api-access-hg87h\") pod \"0a84a8b2-9ebf-4255-938a-fb5c87022755\" (UID: \"0a84a8b2-9ebf-4255-938a-fb5c87022755\") " Sep 10 03:59:39.706993 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:39.706963 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a84a8b2-9ebf-4255-938a-fb5c87022755-kube-api-access-hg87h" (OuterVolumeSpecName: "kube-api-access-hg87h") pod "0a84a8b2-9ebf-4255-938a-fb5c87022755" (UID: "0a84a8b2-9ebf-4255-938a-fb5c87022755"). InnerVolumeSpecName "kube-api-access-hg87h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 03:59:39.806316 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:39.806254 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hg87h\" (UniqueName: \"kubernetes.io/projected/0a84a8b2-9ebf-4255-938a-fb5c87022755-kube-api-access-hg87h\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 03:59:40.396991 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:40.396951 2562 generic.go:356] "Generic (PLEG): container finished" podID="0a84a8b2-9ebf-4255-938a-fb5c87022755" containerID="831f9e7a8490905903551c5b1d8176c6efb526e6539c8b147bc5b065a071b185" exitCode=0 Sep 10 03:59:40.396991 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:40.396990 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-9wlgr" Sep 10 03:59:40.397213 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:40.397028 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-9wlgr" event={"ID":"0a84a8b2-9ebf-4255-938a-fb5c87022755","Type":"ContainerDied","Data":"831f9e7a8490905903551c5b1d8176c6efb526e6539c8b147bc5b065a071b185"} Sep 10 03:59:40.397213 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:40.397068 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-9wlgr" event={"ID":"0a84a8b2-9ebf-4255-938a-fb5c87022755","Type":"ContainerDied","Data":"01e7a1433ea0c2e971d96a954526a817fad2f1e9f57586e2178f9cf3d5d1bb9b"} Sep 10 03:59:40.397213 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:40.397086 2562 scope.go:117] "RemoveContainer" containerID="831f9e7a8490905903551c5b1d8176c6efb526e6539c8b147bc5b065a071b185" Sep 10 03:59:40.405478 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:40.405457 2562 scope.go:117] "RemoveContainer" containerID="831f9e7a8490905903551c5b1d8176c6efb526e6539c8b147bc5b065a071b185" Sep 10 03:59:40.405789 ip-10-0-132-74 kubenswrapper[2562]: E0910 03:59:40.405768 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"831f9e7a8490905903551c5b1d8176c6efb526e6539c8b147bc5b065a071b185\": container with ID starting with 831f9e7a8490905903551c5b1d8176c6efb526e6539c8b147bc5b065a071b185 not found: ID does not exist" containerID="831f9e7a8490905903551c5b1d8176c6efb526e6539c8b147bc5b065a071b185" Sep 10 03:59:40.405848 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:40.405799 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"831f9e7a8490905903551c5b1d8176c6efb526e6539c8b147bc5b065a071b185"} err="failed to get container status \"831f9e7a8490905903551c5b1d8176c6efb526e6539c8b147bc5b065a071b185\": rpc error: code = NotFound desc = could not find container \"831f9e7a8490905903551c5b1d8176c6efb526e6539c8b147bc5b065a071b185\": container with ID starting with 831f9e7a8490905903551c5b1d8176c6efb526e6539c8b147bc5b065a071b185 not found: ID does not exist" Sep 10 03:59:40.416340 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:40.416306 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-9wlgr"] Sep 10 03:59:40.419822 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:40.419794 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-9wlgr"] Sep 10 03:59:41.604518 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:41.604473 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a84a8b2-9ebf-4255-938a-fb5c87022755" path="/var/lib/kubelet/pods/0a84a8b2-9ebf-4255-938a-fb5c87022755/volumes" Sep 10 03:59:47.375897 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:47.375800 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-d2rtt" Sep 10 03:59:49.694143 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:49.694090 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:59:49.793730 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:49.793697 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 03:59:50.151389 ip-10-0-132-74 kubenswrapper[2562]: I0910 03:59:50.151347 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 04:00:02.124148 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:02.124113 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 04:00:02.371015 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:02.370976 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-5bh2c"] Sep 10 04:00:02.371342 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:02.371329 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a84a8b2-9ebf-4255-938a-fb5c87022755" containerName="authorino" Sep 10 04:00:02.371399 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:02.371344 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a84a8b2-9ebf-4255-938a-fb5c87022755" containerName="authorino" Sep 10 04:00:02.371445 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:02.371401 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a84a8b2-9ebf-4255-938a-fb5c87022755" containerName="authorino" Sep 10 04:00:02.375665 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:02.375620 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-5bh2c" Sep 10 04:00:02.378746 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:02.378720 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Sep 10 04:00:02.378889 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:02.378754 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-8xm54\"" Sep 10 04:00:02.381339 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:02.381314 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-5bh2c"] Sep 10 04:00:02.416947 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:02.416904 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 04:00:02.466389 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:02.466357 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-5bh2c"] Sep 10 04:00:02.466634 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:00:02.466616 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-mmplg tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-5bh2c" podUID="0007bcdb-db4c-4faa-88fa-dc4395b07c6b" Sep 10 04:00:02.499570 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:02.499538 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-5bh2c" Sep 10 04:00:02.506125 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:02.506098 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-5bh2c" Sep 10 04:00:02.508622 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:02.508603 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmplg\" (UniqueName: \"kubernetes.io/projected/0007bcdb-db4c-4faa-88fa-dc4395b07c6b-kube-api-access-mmplg\") pod \"authorino-6d4dfc649f-5bh2c\" (UID: \"0007bcdb-db4c-4faa-88fa-dc4395b07c6b\") " pod="kuadrant-system/authorino-6d4dfc649f-5bh2c" Sep 10 04:00:02.508693 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:02.508635 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0007bcdb-db4c-4faa-88fa-dc4395b07c6b-tls-cert\") pod \"authorino-6d4dfc649f-5bh2c\" (UID: \"0007bcdb-db4c-4faa-88fa-dc4395b07c6b\") " pod="kuadrant-system/authorino-6d4dfc649f-5bh2c" Sep 10 04:00:02.609525 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:02.609485 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0007bcdb-db4c-4faa-88fa-dc4395b07c6b-tls-cert\") pod \"authorino-6d4dfc649f-5bh2c\" (UID: \"0007bcdb-db4c-4faa-88fa-dc4395b07c6b\") " pod="kuadrant-system/authorino-6d4dfc649f-5bh2c" Sep 10 04:00:02.609727 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:02.609621 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmplg\" (UniqueName: \"kubernetes.io/projected/0007bcdb-db4c-4faa-88fa-dc4395b07c6b-kube-api-access-mmplg\") pod \"authorino-6d4dfc649f-5bh2c\" (UID: \"0007bcdb-db4c-4faa-88fa-dc4395b07c6b\") " pod="kuadrant-system/authorino-6d4dfc649f-5bh2c" Sep 10 04:00:02.611957 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:02.611928 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0007bcdb-db4c-4faa-88fa-dc4395b07c6b-tls-cert\") pod \"authorino-6d4dfc649f-5bh2c\" (UID: \"0007bcdb-db4c-4faa-88fa-dc4395b07c6b\") " pod="kuadrant-system/authorino-6d4dfc649f-5bh2c" Sep 10 04:00:02.618143 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:02.618116 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmplg\" (UniqueName: \"kubernetes.io/projected/0007bcdb-db4c-4faa-88fa-dc4395b07c6b-kube-api-access-mmplg\") pod \"authorino-6d4dfc649f-5bh2c\" (UID: \"0007bcdb-db4c-4faa-88fa-dc4395b07c6b\") " pod="kuadrant-system/authorino-6d4dfc649f-5bh2c" Sep 10 04:00:02.710411 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:02.710372 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mmplg\" (UniqueName: \"kubernetes.io/projected/0007bcdb-db4c-4faa-88fa-dc4395b07c6b-kube-api-access-mmplg\") pod \"0007bcdb-db4c-4faa-88fa-dc4395b07c6b\" (UID: \"0007bcdb-db4c-4faa-88fa-dc4395b07c6b\") " Sep 10 04:00:02.710628 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:02.710434 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0007bcdb-db4c-4faa-88fa-dc4395b07c6b-tls-cert\") pod \"0007bcdb-db4c-4faa-88fa-dc4395b07c6b\" (UID: \"0007bcdb-db4c-4faa-88fa-dc4395b07c6b\") " Sep 10 04:00:02.712589 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:02.712560 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0007bcdb-db4c-4faa-88fa-dc4395b07c6b-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "0007bcdb-db4c-4faa-88fa-dc4395b07c6b" (UID: "0007bcdb-db4c-4faa-88fa-dc4395b07c6b"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 04:00:02.712711 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:02.712689 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0007bcdb-db4c-4faa-88fa-dc4395b07c6b-kube-api-access-mmplg" (OuterVolumeSpecName: "kube-api-access-mmplg") pod "0007bcdb-db4c-4faa-88fa-dc4395b07c6b" (UID: "0007bcdb-db4c-4faa-88fa-dc4395b07c6b"). InnerVolumeSpecName "kube-api-access-mmplg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 04:00:02.811715 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:02.811674 2562 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0007bcdb-db4c-4faa-88fa-dc4395b07c6b-tls-cert\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 04:00:02.811715 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:02.811707 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmplg\" (UniqueName: \"kubernetes.io/projected/0007bcdb-db4c-4faa-88fa-dc4395b07c6b-kube-api-access-mmplg\") on node \"ip-10-0-132-74.ec2.internal\" DevicePath \"\"" Sep 10 04:00:03.503417 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:03.503383 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-5bh2c" Sep 10 04:00:03.532230 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:03.532194 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-5bh2c"] Sep 10 04:00:03.536364 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:03.536330 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-5bh2c"] Sep 10 04:00:03.605329 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:03.605278 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0007bcdb-db4c-4faa-88fa-dc4395b07c6b" path="/var/lib/kubelet/pods/0007bcdb-db4c-4faa-88fa-dc4395b07c6b/volumes" Sep 10 04:00:10.454283 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:10.454245 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 04:00:10.523586 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:10.523526 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 04:00:10.609594 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:10.609552 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 04:00:13.960640 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:13.960603 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 04:00:14.054699 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:14.054660 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 04:00:46.998956 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:46.998916 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 04:00:47.099148 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:47.099106 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 04:00:48.458915 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:48.458886 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 04:00:48.561023 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:48.560989 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 04:00:48.744722 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:48.744628 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 04:00:48.810432 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:48.810398 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 04:00:49.094565 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:49.094472 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 04:00:49.194836 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:49.194800 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 04:00:49.200249 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:49.200215 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-hfppn"] Sep 10 04:00:49.203653 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:49.203631 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hfppn" Sep 10 04:00:49.205953 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:49.205932 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Sep 10 04:00:49.214838 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:49.214810 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-hfppn"] Sep 10 04:00:49.321561 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:49.321520 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d99147a-e8d3-4a02-b496-8d2cfeed1684-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-hfppn\" (UID: \"2d99147a-e8d3-4a02-b496-8d2cfeed1684\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hfppn" Sep 10 04:00:49.321756 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:49.321597 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2km5\" (UniqueName: \"kubernetes.io/projected/2d99147a-e8d3-4a02-b496-8d2cfeed1684-kube-api-access-x2km5\") pod \"payload-pre-processing-59fbc47bf8-hfppn\" (UID: \"2d99147a-e8d3-4a02-b496-8d2cfeed1684\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hfppn" Sep 10 04:00:49.340245 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:49.340207 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d2rtt"] Sep 10 04:00:49.423033 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:49.422996 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2km5\" (UniqueName: \"kubernetes.io/projected/2d99147a-e8d3-4a02-b496-8d2cfeed1684-kube-api-access-x2km5\") pod \"payload-pre-processing-59fbc47bf8-hfppn\" (UID: \"2d99147a-e8d3-4a02-b496-8d2cfeed1684\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hfppn" Sep 10 04:00:49.423211 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:49.423065 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d99147a-e8d3-4a02-b496-8d2cfeed1684-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-hfppn\" (UID: \"2d99147a-e8d3-4a02-b496-8d2cfeed1684\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hfppn" Sep 10 04:00:49.424024 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:49.424001 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d99147a-e8d3-4a02-b496-8d2cfeed1684-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-hfppn\" (UID: \"2d99147a-e8d3-4a02-b496-8d2cfeed1684\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hfppn" Sep 10 04:00:49.431028 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:49.431002 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2km5\" (UniqueName: \"kubernetes.io/projected/2d99147a-e8d3-4a02-b496-8d2cfeed1684-kube-api-access-x2km5\") pod \"payload-pre-processing-59fbc47bf8-hfppn\" (UID: \"2d99147a-e8d3-4a02-b496-8d2cfeed1684\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hfppn" Sep 10 04:00:49.514403 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:49.514363 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hfppn" Sep 10 04:00:49.785999 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:49.785918 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-hfppn"] Sep 10 04:00:49.789556 ip-10-0-132-74 kubenswrapper[2562]: W0910 04:00:49.789526 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d99147a_e8d3_4a02_b496_8d2cfeed1684.slice/crio-1469bf149548d2ca70835d6d630e09bbebbc36eaa10b32c5a7cd696a18aff6f5 WatchSource:0}: Error finding container 1469bf149548d2ca70835d6d630e09bbebbc36eaa10b32c5a7cd696a18aff6f5: Status 404 returned error can't find the container with id 1469bf149548d2ca70835d6d630e09bbebbc36eaa10b32c5a7cd696a18aff6f5 Sep 10 04:00:50.720089 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:50.720036 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hfppn" event={"ID":"2d99147a-e8d3-4a02-b496-8d2cfeed1684","Type":"ContainerStarted","Data":"1469bf149548d2ca70835d6d630e09bbebbc36eaa10b32c5a7cd696a18aff6f5"} Sep 10 04:00:51.951052 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:51.951012 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 04:00:52.731902 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:52.731858 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hfppn" event={"ID":"2d99147a-e8d3-4a02-b496-8d2cfeed1684","Type":"ContainerStarted","Data":"e601c607891591fcc007fcfe25f619672f0fe8247afa0c438b3ed825a69722d2"} Sep 10 04:00:52.732093 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:52.731994 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hfppn" Sep 10 04:00:52.764534 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:52.764472 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hfppn" podStartSLOduration=1.241575527 podStartE2EDuration="3.764456318s" totalImagesPullingTime="2.522880791s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 04:00:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:00:52.760996275 +0000 UTC m=+779.637059426" watchObservedRunningTime="2026-09-10 04:00:52.764456318 +0000 UTC m=+779.640519469" Sep 10 04:00:59.052005 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:59.051971 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 04:00:59.150305 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:00:59.150269 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 04:01:03.738831 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:01:03.738796 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hfppn" Sep 10 04:01:21.039584 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:01:21.039542 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 04:01:21.443165 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:01:21.443128 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d2rtt"] Sep 10 04:01:37.772388 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:01:37.772353 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 04:01:38.430232 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:01:38.430192 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d2rtt"] Sep 10 04:01:47.263768 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:01:47.263723 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 04:01:48.235178 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:01:48.235134 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d2rtt"] Sep 10 04:02:10.447577 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:10.447533 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-5xwlc"] Sep 10 04:02:10.451662 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:10.451638 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5xwlc" Sep 10 04:02:10.454769 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:10.454740 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 10 04:02:10.454925 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:10.454740 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 10 04:02:10.454925 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:10.454740 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Sep 10 04:02:10.454925 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:10.454740 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4scvv\"" Sep 10 04:02:10.464241 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:10.464216 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-5xwlc"] Sep 10 04:02:10.488569 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:10.488526 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 04:02:10.538826 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:10.538787 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5xwlc\" (UID: \"c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5xwlc" Sep 10 04:02:10.538826 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:10.538831 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5xwlc\" (UID: \"c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5xwlc" Sep 10 04:02:10.539045 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:10.538857 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhs5n\" (UniqueName: \"kubernetes.io/projected/c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495-kube-api-access-hhs5n\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5xwlc\" (UID: \"c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5xwlc" Sep 10 04:02:10.539045 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:10.538919 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5xwlc\" (UID: \"c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5xwlc" Sep 10 04:02:10.539045 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:10.538959 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5xwlc\" (UID: \"c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5xwlc" Sep 10 04:02:10.539045 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:10.538983 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5xwlc\" (UID: \"c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5xwlc" Sep 10 04:02:10.639525 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:10.639482 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5xwlc\" (UID: \"c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5xwlc" Sep 10 04:02:10.639525 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:10.639527 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5xwlc\" (UID: \"c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5xwlc" Sep 10 04:02:10.639815 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:10.639553 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhs5n\" (UniqueName: \"kubernetes.io/projected/c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495-kube-api-access-hhs5n\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5xwlc\" (UID: \"c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5xwlc" Sep 10 04:02:10.639815 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:10.639578 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5xwlc\" (UID: \"c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5xwlc" Sep 10 04:02:10.639815 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:10.639610 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5xwlc\" (UID: \"c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5xwlc" Sep 10 04:02:10.639815 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:10.639658 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5xwlc\" (UID: \"c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5xwlc" Sep 10 04:02:10.640028 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:10.639965 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5xwlc\" (UID: \"c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5xwlc" Sep 10 04:02:10.640082 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:10.640044 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5xwlc\" (UID: \"c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5xwlc" Sep 10 04:02:10.640143 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:10.640119 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5xwlc\" (UID: \"c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5xwlc" Sep 10 04:02:10.641972 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:10.641952 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5xwlc\" (UID: \"c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5xwlc" Sep 10 04:02:10.642388 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:10.642368 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5xwlc\" (UID: \"c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5xwlc" Sep 10 04:02:10.647885 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:10.647863 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhs5n\" (UniqueName: \"kubernetes.io/projected/c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495-kube-api-access-hhs5n\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5xwlc\" (UID: \"c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5xwlc" Sep 10 04:02:10.764108 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:10.764016 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5xwlc" Sep 10 04:02:11.036904 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:11.036876 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-5xwlc"] Sep 10 04:02:11.039624 ip-10-0-132-74 kubenswrapper[2562]: W0910 04:02:11.039596 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0902bfc_4a21_4c3c_b3f6_ae6c6dc6c495.slice/crio-9b0c40f60b08178739ba791cf1123aee7ef7a0815e602295e998da943311101f WatchSource:0}: Error finding container 9b0c40f60b08178739ba791cf1123aee7ef7a0815e602295e998da943311101f: Status 404 returned error can't find the container with id 9b0c40f60b08178739ba791cf1123aee7ef7a0815e602295e998da943311101f Sep 10 04:02:11.098988 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:11.098947 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5xwlc" event={"ID":"c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495","Type":"ContainerStarted","Data":"9b0c40f60b08178739ba791cf1123aee7ef7a0815e602295e998da943311101f"} Sep 10 04:02:11.637103 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:11.637057 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d2rtt"] Sep 10 04:02:13.109720 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:13.109680 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5xwlc" event={"ID":"c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495","Type":"ContainerStarted","Data":"a5a56f04775bb2cbc44cd82c9a159b9f4fb03556727ab645acc059d7f0d5227e"} Sep 10 04:02:13.130051 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:13.129988 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5xwlc" podStartSLOduration=2.122303111 podStartE2EDuration="3.12997029s" totalImagesPullingTime="1.007667179s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 04:02:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:02:13.128918658 +0000 UTC m=+860.004981811" watchObservedRunningTime="2026-09-10 04:02:13.12997029 +0000 UTC m=+860.006033441" Sep 10 04:02:20.764685 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:20.764644 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5xwlc" Sep 10 04:02:20.765077 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:20.764742 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5xwlc" Sep 10 04:02:20.777980 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:20.777950 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5xwlc" Sep 10 04:02:21.156862 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:21.156822 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5xwlc" Sep 10 04:02:21.762019 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:21.761988 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 04:02:23.333961 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:23.333916 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d2rtt"] Sep 10 04:02:40.541892 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:40.541857 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f"] Sep 10 04:02:40.545381 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:40.545352 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:02:40.547761 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:40.547731 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Sep 10 04:02:40.555071 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:40.555038 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f"] Sep 10 04:02:40.592851 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:40.592805 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 04:02:40.616449 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:40.615704 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c222592-85a8-4247-bd05-66be7a758411-dshm\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f\" (UID: \"3c222592-85a8-4247-bd05-66be7a758411\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:02:40.616449 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:40.615761 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c222592-85a8-4247-bd05-66be7a758411-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f\" (UID: \"3c222592-85a8-4247-bd05-66be7a758411\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:02:40.616449 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:40.615949 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnzkw\" (UniqueName: \"kubernetes.io/projected/3c222592-85a8-4247-bd05-66be7a758411-kube-api-access-wnzkw\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f\" (UID: \"3c222592-85a8-4247-bd05-66be7a758411\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:02:40.616449 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:40.616012 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c222592-85a8-4247-bd05-66be7a758411-model-cache\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f\" (UID: \"3c222592-85a8-4247-bd05-66be7a758411\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:02:40.616449 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:40.616098 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c222592-85a8-4247-bd05-66be7a758411-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f\" (UID: \"3c222592-85a8-4247-bd05-66be7a758411\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:02:40.616449 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:40.616173 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c222592-85a8-4247-bd05-66be7a758411-home\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f\" (UID: \"3c222592-85a8-4247-bd05-66be7a758411\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:02:40.717644 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:40.717595 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c222592-85a8-4247-bd05-66be7a758411-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f\" (UID: \"3c222592-85a8-4247-bd05-66be7a758411\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:02:40.717851 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:40.717669 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnzkw\" (UniqueName: \"kubernetes.io/projected/3c222592-85a8-4247-bd05-66be7a758411-kube-api-access-wnzkw\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f\" (UID: \"3c222592-85a8-4247-bd05-66be7a758411\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:02:40.717851 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:40.717698 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c222592-85a8-4247-bd05-66be7a758411-model-cache\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f\" (UID: \"3c222592-85a8-4247-bd05-66be7a758411\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:02:40.717851 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:40.717731 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c222592-85a8-4247-bd05-66be7a758411-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f\" (UID: \"3c222592-85a8-4247-bd05-66be7a758411\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:02:40.717851 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:40.717763 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c222592-85a8-4247-bd05-66be7a758411-home\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f\" (UID: \"3c222592-85a8-4247-bd05-66be7a758411\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:02:40.718056 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:40.717881 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c222592-85a8-4247-bd05-66be7a758411-dshm\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f\" (UID: \"3c222592-85a8-4247-bd05-66be7a758411\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:02:40.718133 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:40.718105 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c222592-85a8-4247-bd05-66be7a758411-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f\" (UID: \"3c222592-85a8-4247-bd05-66be7a758411\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:02:40.718204 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:40.718131 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c222592-85a8-4247-bd05-66be7a758411-model-cache\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f\" (UID: \"3c222592-85a8-4247-bd05-66be7a758411\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:02:40.718204 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:40.718166 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c222592-85a8-4247-bd05-66be7a758411-home\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f\" (UID: \"3c222592-85a8-4247-bd05-66be7a758411\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:02:40.720185 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:40.720163 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c222592-85a8-4247-bd05-66be7a758411-dshm\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f\" (UID: \"3c222592-85a8-4247-bd05-66be7a758411\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:02:40.720322 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:40.720240 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c222592-85a8-4247-bd05-66be7a758411-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f\" (UID: \"3c222592-85a8-4247-bd05-66be7a758411\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:02:40.725506 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:40.725487 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnzkw\" (UniqueName: \"kubernetes.io/projected/3c222592-85a8-4247-bd05-66be7a758411-kube-api-access-wnzkw\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f\" (UID: \"3c222592-85a8-4247-bd05-66be7a758411\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:02:40.878520 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:40.878414 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:02:41.148578 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:41.148546 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f"] Sep 10 04:02:41.152517 ip-10-0-132-74 kubenswrapper[2562]: W0910 04:02:41.152473 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c222592_85a8_4247_bd05_66be7a758411.slice/crio-ef540569d0abf4f4ab78c2e6f213f4d8c0fb065e4500b11adc98a2ec29c5cc71 WatchSource:0}: Error finding container ef540569d0abf4f4ab78c2e6f213f4d8c0fb065e4500b11adc98a2ec29c5cc71: Status 404 returned error can't find the container with id ef540569d0abf4f4ab78c2e6f213f4d8c0fb065e4500b11adc98a2ec29c5cc71 Sep 10 04:02:41.237703 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:41.237658 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" event={"ID":"3c222592-85a8-4247-bd05-66be7a758411","Type":"ContainerStarted","Data":"ef540569d0abf4f4ab78c2e6f213f4d8c0fb065e4500b11adc98a2ec29c5cc71"} Sep 10 04:02:42.244094 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:42.244053 2562 generic.go:356] "Generic (PLEG): container finished" podID="3c222592-85a8-4247-bd05-66be7a758411" containerID="22017748c0f87626763d6052da1b125fd5278f24ca7a05b7f1a5077e525231bb" exitCode=0 Sep 10 04:02:42.244616 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:42.244140 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" event={"ID":"3c222592-85a8-4247-bd05-66be7a758411","Type":"ContainerDied","Data":"22017748c0f87626763d6052da1b125fd5278f24ca7a05b7f1a5077e525231bb"} Sep 10 04:02:42.244616 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:42.244452 2562 scope.go:117] "RemoveContainer" containerID="22017748c0f87626763d6052da1b125fd5278f24ca7a05b7f1a5077e525231bb" Sep 10 04:02:42.441275 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:42.441230 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d2rtt"] Sep 10 04:02:43.251105 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:43.251072 2562 generic.go:356] "Generic (PLEG): container finished" podID="3c222592-85a8-4247-bd05-66be7a758411" containerID="e200f77dda5e389563891fb3ba4fd42da304f1a110510120b4611cdb64cab9b5" exitCode=0 Sep 10 04:02:43.251557 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:43.251162 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" event={"ID":"3c222592-85a8-4247-bd05-66be7a758411","Type":"ContainerDied","Data":"e200f77dda5e389563891fb3ba4fd42da304f1a110510120b4611cdb64cab9b5"} Sep 10 04:02:43.251557 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:43.251223 2562 scope.go:117] "RemoveContainer" containerID="22017748c0f87626763d6052da1b125fd5278f24ca7a05b7f1a5077e525231bb" Sep 10 04:02:43.251557 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:43.251446 2562 scope.go:117] "RemoveContainer" containerID="e200f77dda5e389563891fb3ba4fd42da304f1a110510120b4611cdb64cab9b5" Sep 10 04:02:43.251696 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:02:43.251663 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f_llm(3c222592-85a8-4247-bd05-66be7a758411)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" podUID="3c222592-85a8-4247-bd05-66be7a758411" Sep 10 04:02:44.257585 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:44.257554 2562 scope.go:117] "RemoveContainer" containerID="e200f77dda5e389563891fb3ba4fd42da304f1a110510120b4611cdb64cab9b5" Sep 10 04:02:44.257959 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:02:44.257728 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f_llm(3c222592-85a8-4247-bd05-66be7a758411)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" podUID="3c222592-85a8-4247-bd05-66be7a758411" Sep 10 04:02:50.879201 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:50.879108 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:02:50.879201 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:50.879172 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:02:50.879201 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:50.879188 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:02:50.879201 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:50.879202 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:02:50.879689 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:50.879568 2562 scope.go:117] "RemoveContainer" containerID="e200f77dda5e389563891fb3ba4fd42da304f1a110510120b4611cdb64cab9b5" Sep 10 04:02:50.879752 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:02:50.879732 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f_llm(3c222592-85a8-4247-bd05-66be7a758411)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" podUID="3c222592-85a8-4247-bd05-66be7a758411" Sep 10 04:02:52.333677 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:52.333632 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8"] Sep 10 04:02:52.337029 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:52.337001 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:02:52.339232 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:52.339209 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Sep 10 04:02:52.345860 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:52.345835 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8"] Sep 10 04:02:52.375826 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:52.375782 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 04:02:52.419599 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:52.419557 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/638aee00-aa04-44c1-87a6-c1e179b7e82b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-v58d8\" (UID: \"638aee00-aa04-44c1-87a6-c1e179b7e82b\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:02:52.419784 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:52.419614 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/638aee00-aa04-44c1-87a6-c1e179b7e82b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-v58d8\" (UID: \"638aee00-aa04-44c1-87a6-c1e179b7e82b\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:02:52.419784 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:52.419634 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/638aee00-aa04-44c1-87a6-c1e179b7e82b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-v58d8\" (UID: \"638aee00-aa04-44c1-87a6-c1e179b7e82b\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:02:52.419784 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:52.419719 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wxll\" (UniqueName: \"kubernetes.io/projected/638aee00-aa04-44c1-87a6-c1e179b7e82b-kube-api-access-2wxll\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-v58d8\" (UID: \"638aee00-aa04-44c1-87a6-c1e179b7e82b\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:02:52.419930 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:52.419815 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/638aee00-aa04-44c1-87a6-c1e179b7e82b-home\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-v58d8\" (UID: \"638aee00-aa04-44c1-87a6-c1e179b7e82b\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:02:52.419930 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:52.419846 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/638aee00-aa04-44c1-87a6-c1e179b7e82b-dshm\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-v58d8\" (UID: \"638aee00-aa04-44c1-87a6-c1e179b7e82b\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:02:52.521155 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:52.521101 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/638aee00-aa04-44c1-87a6-c1e179b7e82b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-v58d8\" (UID: \"638aee00-aa04-44c1-87a6-c1e179b7e82b\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:02:52.521155 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:52.521167 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/638aee00-aa04-44c1-87a6-c1e179b7e82b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-v58d8\" (UID: \"638aee00-aa04-44c1-87a6-c1e179b7e82b\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:02:52.521414 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:52.521185 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/638aee00-aa04-44c1-87a6-c1e179b7e82b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-v58d8\" (UID: \"638aee00-aa04-44c1-87a6-c1e179b7e82b\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:02:52.521414 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:52.521217 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wxll\" (UniqueName: \"kubernetes.io/projected/638aee00-aa04-44c1-87a6-c1e179b7e82b-kube-api-access-2wxll\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-v58d8\" (UID: \"638aee00-aa04-44c1-87a6-c1e179b7e82b\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:02:52.521414 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:52.521263 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/638aee00-aa04-44c1-87a6-c1e179b7e82b-home\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-v58d8\" (UID: \"638aee00-aa04-44c1-87a6-c1e179b7e82b\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:02:52.521414 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:52.521283 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/638aee00-aa04-44c1-87a6-c1e179b7e82b-dshm\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-v58d8\" (UID: \"638aee00-aa04-44c1-87a6-c1e179b7e82b\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:02:52.521633 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:52.521606 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/638aee00-aa04-44c1-87a6-c1e179b7e82b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-v58d8\" (UID: \"638aee00-aa04-44c1-87a6-c1e179b7e82b\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:02:52.521689 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:52.521654 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/638aee00-aa04-44c1-87a6-c1e179b7e82b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-v58d8\" (UID: \"638aee00-aa04-44c1-87a6-c1e179b7e82b\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:02:52.521732 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:52.521710 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/638aee00-aa04-44c1-87a6-c1e179b7e82b-home\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-v58d8\" (UID: \"638aee00-aa04-44c1-87a6-c1e179b7e82b\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:02:52.523691 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:52.523669 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/638aee00-aa04-44c1-87a6-c1e179b7e82b-dshm\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-v58d8\" (UID: \"638aee00-aa04-44c1-87a6-c1e179b7e82b\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:02:52.523985 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:52.523962 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/638aee00-aa04-44c1-87a6-c1e179b7e82b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-v58d8\" (UID: \"638aee00-aa04-44c1-87a6-c1e179b7e82b\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:02:52.528989 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:52.528970 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wxll\" (UniqueName: \"kubernetes.io/projected/638aee00-aa04-44c1-87a6-c1e179b7e82b-kube-api-access-2wxll\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-v58d8\" (UID: \"638aee00-aa04-44c1-87a6-c1e179b7e82b\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:02:52.647359 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:52.647321 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:02:52.921115 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:52.921089 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8"] Sep 10 04:02:52.922988 ip-10-0-132-74 kubenswrapper[2562]: W0910 04:02:52.922961 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod638aee00_aa04_44c1_87a6_c1e179b7e82b.slice/crio-0a6f64231b4057658e53af8eec2665b96660017127b879f635543d1f5826e820 WatchSource:0}: Error finding container 0a6f64231b4057658e53af8eec2665b96660017127b879f635543d1f5826e820: Status 404 returned error can't find the container with id 0a6f64231b4057658e53af8eec2665b96660017127b879f635543d1f5826e820 Sep 10 04:02:53.299940 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:53.299906 2562 generic.go:356] "Generic (PLEG): container finished" podID="638aee00-aa04-44c1-87a6-c1e179b7e82b" containerID="f224d64a813b36fe64e6008ae631585ba68713d7cbcbdfd76d99492566bb4cfb" exitCode=0 Sep 10 04:02:53.300121 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:53.299955 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" event={"ID":"638aee00-aa04-44c1-87a6-c1e179b7e82b","Type":"ContainerDied","Data":"f224d64a813b36fe64e6008ae631585ba68713d7cbcbdfd76d99492566bb4cfb"} Sep 10 04:02:53.300121 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:53.299985 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" event={"ID":"638aee00-aa04-44c1-87a6-c1e179b7e82b","Type":"ContainerStarted","Data":"0a6f64231b4057658e53af8eec2665b96660017127b879f635543d1f5826e820"} Sep 10 04:02:53.300217 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:53.300206 2562 scope.go:117] "RemoveContainer" containerID="f224d64a813b36fe64e6008ae631585ba68713d7cbcbdfd76d99492566bb4cfb" Sep 10 04:02:53.581229 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:53.581202 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hf7wl_50451d7c-07fb-4641-a36f-d142e72b5e81/console-operator/2.log" Sep 10 04:02:53.583382 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:53.583357 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hf7wl_50451d7c-07fb-4641-a36f-d142e72b5e81/console-operator/2.log" Sep 10 04:02:53.586909 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:53.586886 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fl7hj_8459a996-8d64-4c23-9e92-1a5bcca253b5/ovn-acl-logging/0.log" Sep 10 04:02:53.588835 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:53.588802 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fl7hj_8459a996-8d64-4c23-9e92-1a5bcca253b5/ovn-acl-logging/0.log" Sep 10 04:02:54.306669 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:54.306635 2562 generic.go:356] "Generic (PLEG): container finished" podID="638aee00-aa04-44c1-87a6-c1e179b7e82b" containerID="b73f7009fa0fddefa63b7e2a1ccbe6b68875dbf0bbfdf384d6a997f20b81ff48" exitCode=0 Sep 10 04:02:54.306846 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:54.306712 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" event={"ID":"638aee00-aa04-44c1-87a6-c1e179b7e82b","Type":"ContainerDied","Data":"b73f7009fa0fddefa63b7e2a1ccbe6b68875dbf0bbfdf384d6a997f20b81ff48"} Sep 10 04:02:54.306846 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:54.306747 2562 scope.go:117] "RemoveContainer" containerID="f224d64a813b36fe64e6008ae631585ba68713d7cbcbdfd76d99492566bb4cfb" Sep 10 04:02:54.307002 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:54.306985 2562 scope.go:117] "RemoveContainer" containerID="b73f7009fa0fddefa63b7e2a1ccbe6b68875dbf0bbfdf384d6a997f20b81ff48" Sep 10 04:02:54.307203 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:02:54.307182 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-v58d8_llm(638aee00-aa04-44c1-87a6-c1e179b7e82b)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" podUID="638aee00-aa04-44c1-87a6-c1e179b7e82b" Sep 10 04:02:54.634123 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:54.634027 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d2rtt"] Sep 10 04:02:55.316338 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:02:55.316311 2562 scope.go:117] "RemoveContainer" containerID="b73f7009fa0fddefa63b7e2a1ccbe6b68875dbf0bbfdf384d6a997f20b81ff48" Sep 10 04:02:55.316528 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:02:55.316504 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-v58d8_llm(638aee00-aa04-44c1-87a6-c1e179b7e82b)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" podUID="638aee00-aa04-44c1-87a6-c1e179b7e82b" Sep 10 04:03:02.647417 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:02.647376 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:03:02.647417 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:02.647428 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:03:02.647887 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:02.647438 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:03:02.647887 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:02.647448 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:03:02.647887 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:02.647769 2562 scope.go:117] "RemoveContainer" containerID="b73f7009fa0fddefa63b7e2a1ccbe6b68875dbf0bbfdf384d6a997f20b81ff48" Sep 10 04:03:02.647984 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:03:02.647934 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-v58d8_llm(638aee00-aa04-44c1-87a6-c1e179b7e82b)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" podUID="638aee00-aa04-44c1-87a6-c1e179b7e82b" Sep 10 04:03:04.599060 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:04.599019 2562 scope.go:117] "RemoveContainer" containerID="e200f77dda5e389563891fb3ba4fd42da304f1a110510120b4611cdb64cab9b5" Sep 10 04:03:05.371458 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:05.371422 2562 generic.go:356] "Generic (PLEG): container finished" podID="3c222592-85a8-4247-bd05-66be7a758411" containerID="d7b31cc0f471798e758df38c84fc01da510b90c04c3dd3eb55801a942888a818" exitCode=0 Sep 10 04:03:05.371644 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:05.371493 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" event={"ID":"3c222592-85a8-4247-bd05-66be7a758411","Type":"ContainerDied","Data":"d7b31cc0f471798e758df38c84fc01da510b90c04c3dd3eb55801a942888a818"} Sep 10 04:03:05.371644 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:05.371554 2562 scope.go:117] "RemoveContainer" containerID="e200f77dda5e389563891fb3ba4fd42da304f1a110510120b4611cdb64cab9b5" Sep 10 04:03:05.371927 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:05.371908 2562 scope.go:117] "RemoveContainer" containerID="d7b31cc0f471798e758df38c84fc01da510b90c04c3dd3eb55801a942888a818" Sep 10 04:03:05.372141 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:03:05.372117 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f_llm(3c222592-85a8-4247-bd05-66be7a758411)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" podUID="3c222592-85a8-4247-bd05-66be7a758411" Sep 10 04:03:09.171602 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:09.171555 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 04:03:10.879283 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:10.879246 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:03:10.879737 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:10.879343 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:03:10.879737 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:10.879362 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:03:10.879737 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:10.879375 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:03:10.879841 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:10.879804 2562 scope.go:117] "RemoveContainer" containerID="d7b31cc0f471798e758df38c84fc01da510b90c04c3dd3eb55801a942888a818" Sep 10 04:03:10.880035 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:03:10.880013 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f_llm(3c222592-85a8-4247-bd05-66be7a758411)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" podUID="3c222592-85a8-4247-bd05-66be7a758411" Sep 10 04:03:11.135349 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:11.135259 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 04:03:17.013638 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:17.013605 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 04:03:17.097496 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:17.097455 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 04:03:17.599627 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:17.599598 2562 scope.go:117] "RemoveContainer" containerID="b73f7009fa0fddefa63b7e2a1ccbe6b68875dbf0bbfdf384d6a997f20b81ff48" Sep 10 04:03:18.434218 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:18.434179 2562 generic.go:356] "Generic (PLEG): container finished" podID="638aee00-aa04-44c1-87a6-c1e179b7e82b" containerID="1027e96368c9197715a7615c9f5c7bd321ed25c367bbecfe690183973841426b" exitCode=0 Sep 10 04:03:18.434653 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:18.434246 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" event={"ID":"638aee00-aa04-44c1-87a6-c1e179b7e82b","Type":"ContainerDied","Data":"1027e96368c9197715a7615c9f5c7bd321ed25c367bbecfe690183973841426b"} Sep 10 04:03:18.434653 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:18.434311 2562 scope.go:117] "RemoveContainer" containerID="b73f7009fa0fddefa63b7e2a1ccbe6b68875dbf0bbfdf384d6a997f20b81ff48" Sep 10 04:03:18.434741 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:18.434660 2562 scope.go:117] "RemoveContainer" containerID="1027e96368c9197715a7615c9f5c7bd321ed25c367bbecfe690183973841426b" Sep 10 04:03:18.434872 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:03:18.434854 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-v58d8_llm(638aee00-aa04-44c1-87a6-c1e179b7e82b)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" podUID="638aee00-aa04-44c1-87a6-c1e179b7e82b" Sep 10 04:03:22.599227 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:22.599184 2562 scope.go:117] "RemoveContainer" containerID="d7b31cc0f471798e758df38c84fc01da510b90c04c3dd3eb55801a942888a818" Sep 10 04:03:22.599703 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:03:22.599422 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f_llm(3c222592-85a8-4247-bd05-66be7a758411)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" podUID="3c222592-85a8-4247-bd05-66be7a758411" Sep 10 04:03:22.647827 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:22.647776 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:03:22.647974 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:22.647845 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:03:22.647974 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:22.647856 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:03:22.647974 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:22.647865 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:03:22.648233 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:22.648219 2562 scope.go:117] "RemoveContainer" containerID="1027e96368c9197715a7615c9f5c7bd321ed25c367bbecfe690183973841426b" Sep 10 04:03:22.648431 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:03:22.648412 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-v58d8_llm(638aee00-aa04-44c1-87a6-c1e179b7e82b)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" podUID="638aee00-aa04-44c1-87a6-c1e179b7e82b" Sep 10 04:03:36.598834 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:36.598792 2562 scope.go:117] "RemoveContainer" containerID="d7b31cc0f471798e758df38c84fc01da510b90c04c3dd3eb55801a942888a818" Sep 10 04:03:37.525339 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:37.525283 2562 generic.go:356] "Generic (PLEG): container finished" podID="3c222592-85a8-4247-bd05-66be7a758411" containerID="0a3f85199280bc23de8dd3610d9aea9434e4a92e74ed9ee5000553cea99fc556" exitCode=0 Sep 10 04:03:37.525515 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:37.525356 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" event={"ID":"3c222592-85a8-4247-bd05-66be7a758411","Type":"ContainerDied","Data":"0a3f85199280bc23de8dd3610d9aea9434e4a92e74ed9ee5000553cea99fc556"} Sep 10 04:03:37.525515 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:37.525419 2562 scope.go:117] "RemoveContainer" containerID="d7b31cc0f471798e758df38c84fc01da510b90c04c3dd3eb55801a942888a818" Sep 10 04:03:37.525816 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:37.525796 2562 scope.go:117] "RemoveContainer" containerID="0a3f85199280bc23de8dd3610d9aea9434e4a92e74ed9ee5000553cea99fc556" Sep 10 04:03:37.526031 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:03:37.526008 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f_llm(3c222592-85a8-4247-bd05-66be7a758411)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" podUID="3c222592-85a8-4247-bd05-66be7a758411" Sep 10 04:03:37.599727 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:37.599685 2562 scope.go:117] "RemoveContainer" containerID="1027e96368c9197715a7615c9f5c7bd321ed25c367bbecfe690183973841426b" Sep 10 04:03:37.600128 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:03:37.599937 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-v58d8_llm(638aee00-aa04-44c1-87a6-c1e179b7e82b)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" podUID="638aee00-aa04-44c1-87a6-c1e179b7e82b" Sep 10 04:03:40.879088 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:40.879027 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:03:40.879574 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:40.879121 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:03:40.879574 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:40.879144 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:03:40.879574 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:40.879163 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:03:40.879574 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:40.879561 2562 scope.go:117] "RemoveContainer" containerID="0a3f85199280bc23de8dd3610d9aea9434e4a92e74ed9ee5000553cea99fc556" Sep 10 04:03:40.879753 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:03:40.879731 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f_llm(3c222592-85a8-4247-bd05-66be7a758411)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" podUID="3c222592-85a8-4247-bd05-66be7a758411" Sep 10 04:03:50.598737 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:50.598691 2562 scope.go:117] "RemoveContainer" containerID="1027e96368c9197715a7615c9f5c7bd321ed25c367bbecfe690183973841426b" Sep 10 04:03:50.599363 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:50.599079 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 04:03:51.593528 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:51.593491 2562 generic.go:356] "Generic (PLEG): container finished" podID="638aee00-aa04-44c1-87a6-c1e179b7e82b" containerID="6b7e2308554bfd35237c766715dad42a6163307b703610231ae9ec3d627521b3" exitCode=0 Sep 10 04:03:51.593737 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:51.593564 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" event={"ID":"638aee00-aa04-44c1-87a6-c1e179b7e82b","Type":"ContainerDied","Data":"6b7e2308554bfd35237c766715dad42a6163307b703610231ae9ec3d627521b3"} Sep 10 04:03:51.593737 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:51.593618 2562 scope.go:117] "RemoveContainer" containerID="1027e96368c9197715a7615c9f5c7bd321ed25c367bbecfe690183973841426b" Sep 10 04:03:51.594012 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:51.593987 2562 scope.go:117] "RemoveContainer" containerID="6b7e2308554bfd35237c766715dad42a6163307b703610231ae9ec3d627521b3" Sep 10 04:03:51.594221 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:03:51.594196 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-v58d8_llm(638aee00-aa04-44c1-87a6-c1e179b7e82b)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" podUID="638aee00-aa04-44c1-87a6-c1e179b7e82b" Sep 10 04:03:52.647752 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:52.647712 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:03:52.648153 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:52.647780 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:03:52.648153 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:52.647793 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:03:52.648153 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:52.647802 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:03:52.648153 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:52.648134 2562 scope.go:117] "RemoveContainer" containerID="6b7e2308554bfd35237c766715dad42a6163307b703610231ae9ec3d627521b3" Sep 10 04:03:52.648377 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:03:52.648349 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-v58d8_llm(638aee00-aa04-44c1-87a6-c1e179b7e82b)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" podUID="638aee00-aa04-44c1-87a6-c1e179b7e82b" Sep 10 04:03:55.599672 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:55.599629 2562 scope.go:117] "RemoveContainer" containerID="0a3f85199280bc23de8dd3610d9aea9434e4a92e74ed9ee5000553cea99fc556" Sep 10 04:03:55.600159 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:03:55.599836 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f_llm(3c222592-85a8-4247-bd05-66be7a758411)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" podUID="3c222592-85a8-4247-bd05-66be7a758411" Sep 10 04:03:57.413800 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:57.413758 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 04:03:57.512641 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:03:57.512604 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 04:04:05.599515 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:04:05.599471 2562 scope.go:117] "RemoveContainer" containerID="6b7e2308554bfd35237c766715dad42a6163307b703610231ae9ec3d627521b3" Sep 10 04:04:05.599913 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:04:05.599699 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-v58d8_llm(638aee00-aa04-44c1-87a6-c1e179b7e82b)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" podUID="638aee00-aa04-44c1-87a6-c1e179b7e82b" Sep 10 04:04:07.599412 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:04:07.599376 2562 scope.go:117] "RemoveContainer" containerID="0a3f85199280bc23de8dd3610d9aea9434e4a92e74ed9ee5000553cea99fc556" Sep 10 04:04:07.599784 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:04:07.599573 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f_llm(3c222592-85a8-4247-bd05-66be7a758411)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" podUID="3c222592-85a8-4247-bd05-66be7a758411" Sep 10 04:04:18.599271 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:04:18.599234 2562 scope.go:117] "RemoveContainer" containerID="6b7e2308554bfd35237c766715dad42a6163307b703610231ae9ec3d627521b3" Sep 10 04:04:18.599701 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:04:18.599508 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-v58d8_llm(638aee00-aa04-44c1-87a6-c1e179b7e82b)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" podUID="638aee00-aa04-44c1-87a6-c1e179b7e82b" Sep 10 04:04:21.598799 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:04:21.598763 2562 scope.go:117] "RemoveContainer" containerID="0a3f85199280bc23de8dd3610d9aea9434e4a92e74ed9ee5000553cea99fc556" Sep 10 04:04:22.744519 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:04:22.744485 2562 generic.go:356] "Generic (PLEG): container finished" podID="3c222592-85a8-4247-bd05-66be7a758411" containerID="a190e7f67a0e6a416a7929ebc80dd0683c79be60d4285b922db20fa1d7788f50" exitCode=0 Sep 10 04:04:22.744915 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:04:22.744560 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" event={"ID":"3c222592-85a8-4247-bd05-66be7a758411","Type":"ContainerDied","Data":"a190e7f67a0e6a416a7929ebc80dd0683c79be60d4285b922db20fa1d7788f50"} Sep 10 04:04:22.744915 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:04:22.744623 2562 scope.go:117] "RemoveContainer" containerID="0a3f85199280bc23de8dd3610d9aea9434e4a92e74ed9ee5000553cea99fc556" Sep 10 04:04:22.745063 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:04:22.745045 2562 scope.go:117] "RemoveContainer" containerID="a190e7f67a0e6a416a7929ebc80dd0683c79be60d4285b922db20fa1d7788f50" Sep 10 04:04:22.745306 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:04:22.745275 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f_llm(3c222592-85a8-4247-bd05-66be7a758411)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" podUID="3c222592-85a8-4247-bd05-66be7a758411" Sep 10 04:04:30.879125 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:04:30.879077 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:04:30.879565 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:04:30.879152 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:04:30.879565 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:04:30.879168 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:04:30.879565 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:04:30.879182 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:04:30.879667 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:04:30.879602 2562 scope.go:117] "RemoveContainer" containerID="a190e7f67a0e6a416a7929ebc80dd0683c79be60d4285b922db20fa1d7788f50" Sep 10 04:04:30.879827 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:04:30.879806 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f_llm(3c222592-85a8-4247-bd05-66be7a758411)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" podUID="3c222592-85a8-4247-bd05-66be7a758411" Sep 10 04:04:31.599521 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:04:31.599487 2562 scope.go:117] "RemoveContainer" containerID="6b7e2308554bfd35237c766715dad42a6163307b703610231ae9ec3d627521b3" Sep 10 04:04:32.799854 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:04:32.799816 2562 generic.go:356] "Generic (PLEG): container finished" podID="638aee00-aa04-44c1-87a6-c1e179b7e82b" containerID="d2631744597226bdcdf01a1de8c98d33dee6a7f1f933778728696fdcac28725e" exitCode=0 Sep 10 04:04:32.800342 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:04:32.799892 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" event={"ID":"638aee00-aa04-44c1-87a6-c1e179b7e82b","Type":"ContainerDied","Data":"d2631744597226bdcdf01a1de8c98d33dee6a7f1f933778728696fdcac28725e"} Sep 10 04:04:32.800342 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:04:32.799950 2562 scope.go:117] "RemoveContainer" containerID="6b7e2308554bfd35237c766715dad42a6163307b703610231ae9ec3d627521b3" Sep 10 04:04:32.800458 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:04:32.800378 2562 scope.go:117] "RemoveContainer" containerID="d2631744597226bdcdf01a1de8c98d33dee6a7f1f933778728696fdcac28725e" Sep 10 04:04:32.800568 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:04:32.800548 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-v58d8_llm(638aee00-aa04-44c1-87a6-c1e179b7e82b)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" podUID="638aee00-aa04-44c1-87a6-c1e179b7e82b" Sep 10 04:04:42.599763 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:04:42.599723 2562 scope.go:117] "RemoveContainer" containerID="a190e7f67a0e6a416a7929ebc80dd0683c79be60d4285b922db20fa1d7788f50" Sep 10 04:04:42.600130 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:04:42.599901 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f_llm(3c222592-85a8-4247-bd05-66be7a758411)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" podUID="3c222592-85a8-4247-bd05-66be7a758411" Sep 10 04:04:42.647896 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:04:42.647845 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:04:42.647896 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:04:42.647910 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:04:42.648111 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:04:42.647925 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:04:42.648111 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:04:42.647934 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:04:42.648315 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:04:42.648283 2562 scope.go:117] "RemoveContainer" containerID="d2631744597226bdcdf01a1de8c98d33dee6a7f1f933778728696fdcac28725e" Sep 10 04:04:42.648485 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:04:42.648466 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-v58d8_llm(638aee00-aa04-44c1-87a6-c1e179b7e82b)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" podUID="638aee00-aa04-44c1-87a6-c1e179b7e82b" Sep 10 04:04:53.602927 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:04:53.602897 2562 scope.go:117] "RemoveContainer" containerID="a190e7f67a0e6a416a7929ebc80dd0683c79be60d4285b922db20fa1d7788f50" Sep 10 04:04:53.603479 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:04:53.603088 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f_llm(3c222592-85a8-4247-bd05-66be7a758411)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" podUID="3c222592-85a8-4247-bd05-66be7a758411" Sep 10 04:04:56.598807 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:04:56.598763 2562 scope.go:117] "RemoveContainer" containerID="d2631744597226bdcdf01a1de8c98d33dee6a7f1f933778728696fdcac28725e" Sep 10 04:04:56.599320 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:04:56.598977 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-v58d8_llm(638aee00-aa04-44c1-87a6-c1e179b7e82b)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" podUID="638aee00-aa04-44c1-87a6-c1e179b7e82b" Sep 10 04:05:04.598924 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:05:04.598888 2562 scope.go:117] "RemoveContainer" containerID="a190e7f67a0e6a416a7929ebc80dd0683c79be60d4285b922db20fa1d7788f50" Sep 10 04:05:04.599346 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:05:04.599080 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f_llm(3c222592-85a8-4247-bd05-66be7a758411)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" podUID="3c222592-85a8-4247-bd05-66be7a758411" Sep 10 04:05:10.599193 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:05:10.599119 2562 scope.go:117] "RemoveContainer" containerID="d2631744597226bdcdf01a1de8c98d33dee6a7f1f933778728696fdcac28725e" Sep 10 04:05:10.599596 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:05:10.599340 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-v58d8_llm(638aee00-aa04-44c1-87a6-c1e179b7e82b)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" podUID="638aee00-aa04-44c1-87a6-c1e179b7e82b" Sep 10 04:05:15.605040 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:05:15.605008 2562 scope.go:117] "RemoveContainer" containerID="a190e7f67a0e6a416a7929ebc80dd0683c79be60d4285b922db20fa1d7788f50" Sep 10 04:05:15.605453 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:05:15.605209 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f_llm(3c222592-85a8-4247-bd05-66be7a758411)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" podUID="3c222592-85a8-4247-bd05-66be7a758411" Sep 10 04:05:21.604474 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:05:21.604442 2562 scope.go:117] "RemoveContainer" containerID="d2631744597226bdcdf01a1de8c98d33dee6a7f1f933778728696fdcac28725e" Sep 10 04:05:21.604873 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:05:21.604627 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-v58d8_llm(638aee00-aa04-44c1-87a6-c1e179b7e82b)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" podUID="638aee00-aa04-44c1-87a6-c1e179b7e82b" Sep 10 04:05:27.599338 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:05:27.599274 2562 scope.go:117] "RemoveContainer" containerID="a190e7f67a0e6a416a7929ebc80dd0683c79be60d4285b922db20fa1d7788f50" Sep 10 04:05:27.599842 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:05:27.599491 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f_llm(3c222592-85a8-4247-bd05-66be7a758411)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" podUID="3c222592-85a8-4247-bd05-66be7a758411" Sep 10 04:05:34.599130 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:05:34.599092 2562 scope.go:117] "RemoveContainer" containerID="d2631744597226bdcdf01a1de8c98d33dee6a7f1f933778728696fdcac28725e" Sep 10 04:05:34.599561 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:05:34.599271 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-v58d8_llm(638aee00-aa04-44c1-87a6-c1e179b7e82b)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" podUID="638aee00-aa04-44c1-87a6-c1e179b7e82b" Sep 10 04:05:40.598764 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:05:40.598727 2562 scope.go:117] "RemoveContainer" containerID="a190e7f67a0e6a416a7929ebc80dd0683c79be60d4285b922db20fa1d7788f50" Sep 10 04:05:40.599159 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:05:40.598932 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f_llm(3c222592-85a8-4247-bd05-66be7a758411)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" podUID="3c222592-85a8-4247-bd05-66be7a758411" Sep 10 04:05:45.603320 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:05:45.599862 2562 scope.go:117] "RemoveContainer" containerID="d2631744597226bdcdf01a1de8c98d33dee6a7f1f933778728696fdcac28725e" Sep 10 04:05:45.603320 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:05:45.600181 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-v58d8_llm(638aee00-aa04-44c1-87a6-c1e179b7e82b)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" podUID="638aee00-aa04-44c1-87a6-c1e179b7e82b" Sep 10 04:05:54.599569 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:05:54.599526 2562 scope.go:117] "RemoveContainer" containerID="a190e7f67a0e6a416a7929ebc80dd0683c79be60d4285b922db20fa1d7788f50" Sep 10 04:05:55.194464 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:05:55.194431 2562 generic.go:356] "Generic (PLEG): container finished" podID="3c222592-85a8-4247-bd05-66be7a758411" containerID="606d636337a7c6642f4640fed2dfbeb5cd707a5d83ce0788f162dbb6a9567205" exitCode=0 Sep 10 04:05:55.194664 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:05:55.194503 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" event={"ID":"3c222592-85a8-4247-bd05-66be7a758411","Type":"ContainerDied","Data":"606d636337a7c6642f4640fed2dfbeb5cd707a5d83ce0788f162dbb6a9567205"} Sep 10 04:05:55.194664 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:05:55.194562 2562 scope.go:117] "RemoveContainer" containerID="a190e7f67a0e6a416a7929ebc80dd0683c79be60d4285b922db20fa1d7788f50" Sep 10 04:05:55.194951 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:05:55.194931 2562 scope.go:117] "RemoveContainer" containerID="606d636337a7c6642f4640fed2dfbeb5cd707a5d83ce0788f162dbb6a9567205" Sep 10 04:05:55.195144 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:05:55.195120 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f_llm(3c222592-85a8-4247-bd05-66be7a758411)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" podUID="3c222592-85a8-4247-bd05-66be7a758411" Sep 10 04:05:58.228670 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:05:58.228634 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 04:05:59.599528 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:05:59.599494 2562 scope.go:117] "RemoveContainer" containerID="d2631744597226bdcdf01a1de8c98d33dee6a7f1f933778728696fdcac28725e" Sep 10 04:06:00.220862 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:00.220823 2562 generic.go:356] "Generic (PLEG): container finished" podID="638aee00-aa04-44c1-87a6-c1e179b7e82b" containerID="77c2cb494206a72cfbd7ea653a7cbab8d8c7b05a3ba0b0c0cb2d90ecc8277bcd" exitCode=0 Sep 10 04:06:00.221076 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:00.220891 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" event={"ID":"638aee00-aa04-44c1-87a6-c1e179b7e82b","Type":"ContainerDied","Data":"77c2cb494206a72cfbd7ea653a7cbab8d8c7b05a3ba0b0c0cb2d90ecc8277bcd"} Sep 10 04:06:00.221076 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:00.220950 2562 scope.go:117] "RemoveContainer" containerID="d2631744597226bdcdf01a1de8c98d33dee6a7f1f933778728696fdcac28725e" Sep 10 04:06:00.221354 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:00.221335 2562 scope.go:117] "RemoveContainer" containerID="77c2cb494206a72cfbd7ea653a7cbab8d8c7b05a3ba0b0c0cb2d90ecc8277bcd" Sep 10 04:06:00.221566 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:06:00.221541 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-v58d8_llm(638aee00-aa04-44c1-87a6-c1e179b7e82b)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" podUID="638aee00-aa04-44c1-87a6-c1e179b7e82b" Sep 10 04:06:00.575840 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:00.575717 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 04:06:00.878782 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:00.878683 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:06:00.878782 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:00.878769 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:06:00.879321 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:00.878794 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:06:00.879321 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:00.878814 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" Sep 10 04:06:00.879321 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:00.879260 2562 scope.go:117] "RemoveContainer" containerID="606d636337a7c6642f4640fed2dfbeb5cd707a5d83ce0788f162dbb6a9567205" Sep 10 04:06:00.879553 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:06:00.879530 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f_llm(3c222592-85a8-4247-bd05-66be7a758411)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" podUID="3c222592-85a8-4247-bd05-66be7a758411" Sep 10 04:06:02.543012 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:02.542979 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f_3c222592-85a8-4247-bd05-66be7a758411/main/5.log" Sep 10 04:06:02.647499 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:02.647442 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:06:02.647499 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:02.647512 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:06:02.647737 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:02.647526 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:06:02.647737 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:02.647535 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" Sep 10 04:06:02.647877 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:02.647863 2562 scope.go:117] "RemoveContainer" containerID="77c2cb494206a72cfbd7ea653a7cbab8d8c7b05a3ba0b0c0cb2d90ecc8277bcd" Sep 10 04:06:02.648064 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:06:02.648047 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-v58d8_llm(638aee00-aa04-44c1-87a6-c1e179b7e82b)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" podUID="638aee00-aa04-44c1-87a6-c1e179b7e82b" Sep 10 04:06:02.661266 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:02.661239 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f_3c222592-85a8-4247-bd05-66be7a758411/main/5.log" Sep 10 04:06:09.907544 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:09.907504 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-797557847b-nb65p_6a0f70b9-e517-4704-8019-12e9cfd9fd80/copy-manifests/0.log" Sep 10 04:06:09.914286 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:09.914253 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-797557847b-nb65p_6a0f70b9-e517-4704-8019-12e9cfd9fd80/manager/0.log" Sep 10 04:06:11.494067 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:11.494038 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-vvdhn_670d97b5-b0f6-4141-a2cf-955aa87e779b/kuadrant-console-plugin/0.log" Sep 10 04:06:11.733657 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:11.733627 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-d2rtt_64f870d7-9485-4051-b64c-ffe5403791c5/limitador/0.log" Sep 10 04:06:12.529199 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:12.529149 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dcb5c9454-wstk2_41b98efd-b6d8-413c-9017-10e71dabaacf/kube-auth-proxy/0.log" Sep 10 04:06:12.598734 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:12.598704 2562 scope.go:117] "RemoveContainer" containerID="606d636337a7c6642f4640fed2dfbeb5cd707a5d83ce0788f162dbb6a9567205" Sep 10 04:06:12.598902 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:06:12.598872 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f_llm(3c222592-85a8-4247-bd05-66be7a758411)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" podUID="3c222592-85a8-4247-bd05-66be7a758411" Sep 10 04:06:12.749814 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:12.749778 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-hfppn_2d99147a-e8d3-4a02-b496-8d2cfeed1684/payload-pre-processing/0.log" Sep 10 04:06:13.692895 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:13.692865 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-5xwlc_c0902bfc-4a21-4c3c-b3f6-ae6c6dc6c495/main/0.log" Sep 10 04:06:13.908125 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:13.908094 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f_3c222592-85a8-4247-bd05-66be7a758411/main/5.log" Sep 10 04:06:14.019241 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:14.019152 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5758cf6949-v58d8_638aee00-aa04-44c1-87a6-c1e179b7e82b/main/5.log" Sep 10 04:06:16.598984 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:16.598949 2562 scope.go:117] "RemoveContainer" containerID="77c2cb494206a72cfbd7ea653a7cbab8d8c7b05a3ba0b0c0cb2d90ecc8277bcd" Sep 10 04:06:16.599398 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:06:16.599112 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-v58d8_llm(638aee00-aa04-44c1-87a6-c1e179b7e82b)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" podUID="638aee00-aa04-44c1-87a6-c1e179b7e82b" Sep 10 04:06:19.549158 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:19.549115 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 04:06:19.647041 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:19.647004 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" limit=1024 Sep 10 04:06:22.576167 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:22.576133 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-gwzk7_348bacd3-9294-48e6-bbf4-60727b9a7bfb/global-pull-secret-syncer/0.log" Sep 10 04:06:22.672967 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:22.672933 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6ds5c_46180a14-d2d5-44da-8ed7-80a47696fcf7/konnectivity-agent/0.log" Sep 10 04:06:22.772355 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:22.772323 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-74.ec2.internal_95b230e05074eeabdbb71cf81104af8d/haproxy/0.log" Sep 10 04:06:26.599478 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:26.599389 2562 scope.go:117] "RemoveContainer" containerID="606d636337a7c6642f4640fed2dfbeb5cd707a5d83ce0788f162dbb6a9567205" Sep 10 04:06:26.599844 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:06:26.599584 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f_llm(3c222592-85a8-4247-bd05-66be7a758411)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" podUID="3c222592-85a8-4247-bd05-66be7a758411" Sep 10 04:06:27.348415 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:27.348390 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-vvdhn_670d97b5-b0f6-4141-a2cf-955aa87e779b/kuadrant-console-plugin/0.log" Sep 10 04:06:27.412448 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:27.412418 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-d2rtt_64f870d7-9485-4051-b64c-ffe5403791c5/limitador/0.log" Sep 10 04:06:27.599196 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:27.599110 2562 scope.go:117] "RemoveContainer" containerID="77c2cb494206a72cfbd7ea653a7cbab8d8c7b05a3ba0b0c0cb2d90ecc8277bcd" Sep 10 04:06:27.599392 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:06:27.599266 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-v58d8_llm(638aee00-aa04-44c1-87a6-c1e179b7e82b)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" podUID="638aee00-aa04-44c1-87a6-c1e179b7e82b" Sep 10 04:06:29.165814 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:29.165775 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-q5p2x_678609c3-b3fa-4beb-953d-d61067e32212/cluster-monitoring-operator/0.log" Sep 10 04:06:29.472119 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:29.472030 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sbhps_3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd/node-exporter/0.log" Sep 10 04:06:29.496175 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:29.496109 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sbhps_3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd/kube-rbac-proxy/0.log" Sep 10 04:06:29.517600 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:29.517574 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sbhps_3af1ec86-b6f3-4d16-bb0e-16cb77a80dfd/init-textfile/0.log" Sep 10 04:06:29.663097 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:29.663068 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_59c7afc9-5012-4dae-8cea-2575633ab869/prometheus/0.log" Sep 10 04:06:29.683439 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:29.683407 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_59c7afc9-5012-4dae-8cea-2575633ab869/config-reloader/0.log" Sep 10 04:06:29.704474 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:29.704446 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_59c7afc9-5012-4dae-8cea-2575633ab869/thanos-sidecar/0.log" Sep 10 04:06:29.724594 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:29.724510 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_59c7afc9-5012-4dae-8cea-2575633ab869/kube-rbac-proxy-web/0.log" Sep 10 04:06:29.746522 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:29.746490 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_59c7afc9-5012-4dae-8cea-2575633ab869/kube-rbac-proxy/0.log" Sep 10 04:06:29.771436 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:29.771407 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_59c7afc9-5012-4dae-8cea-2575633ab869/kube-rbac-proxy-thanos/0.log" Sep 10 04:06:29.799118 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:29.799089 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_59c7afc9-5012-4dae-8cea-2575633ab869/init-config-reloader/0.log" Sep 10 04:06:30.071858 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:30.071774 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65894fbfc9-v8z5g_4d5fd4c5-9178-4ae2-994a-445fa67e6174/thanos-query/0.log" Sep 10 04:06:30.127196 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:30.127163 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65894fbfc9-v8z5g_4d5fd4c5-9178-4ae2-994a-445fa67e6174/kube-rbac-proxy-web/0.log" Sep 10 04:06:30.148443 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:30.148413 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65894fbfc9-v8z5g_4d5fd4c5-9178-4ae2-994a-445fa67e6174/kube-rbac-proxy/0.log" Sep 10 04:06:30.169057 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:30.169022 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65894fbfc9-v8z5g_4d5fd4c5-9178-4ae2-994a-445fa67e6174/prom-label-proxy/0.log" Sep 10 04:06:30.195189 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:30.195163 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65894fbfc9-v8z5g_4d5fd4c5-9178-4ae2-994a-445fa67e6174/kube-rbac-proxy-rules/0.log" Sep 10 04:06:30.214117 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:30.214077 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65894fbfc9-v8z5g_4d5fd4c5-9178-4ae2-994a-445fa67e6174/kube-rbac-proxy-metrics/0.log" Sep 10 04:06:31.713556 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:31.713528 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-2mjrb_aa9e7159-181a-4a1f-afe1-33cd08e9ba6a/networking-console-plugin/0.log" Sep 10 04:06:32.296107 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:32.296077 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hf7wl_50451d7c-07fb-4641-a36f-d142e72b5e81/console-operator/2.log" Sep 10 04:06:32.303057 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:32.303024 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hf7wl_50451d7c-07fb-4641-a36f-d142e72b5e81/console-operator/3.log" Sep 10 04:06:32.653080 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:32.653030 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5qpb6/perf-node-gather-daemonset-524x8"] Sep 10 04:06:32.656887 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:32.656861 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-524x8" Sep 10 04:06:32.659224 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:32.659199 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5qpb6\"/\"kube-root-ca.crt\"" Sep 10 04:06:32.659371 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:32.659202 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5qpb6\"/\"default-dockercfg-fppwb\"" Sep 10 04:06:32.659371 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:32.659203 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5qpb6\"/\"openshift-service-ca.crt\"" Sep 10 04:06:32.663142 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:32.663119 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5qpb6/perf-node-gather-daemonset-524x8"] Sep 10 04:06:32.744638 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:32.744597 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vhbg\" (UniqueName: \"kubernetes.io/projected/22cac32f-e82d-4e46-bbeb-09b8203e9e4e-kube-api-access-5vhbg\") pod \"perf-node-gather-daemonset-524x8\" (UID: \"22cac32f-e82d-4e46-bbeb-09b8203e9e4e\") " pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-524x8" Sep 10 04:06:32.744638 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:32.744648 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/22cac32f-e82d-4e46-bbeb-09b8203e9e4e-lib-modules\") pod \"perf-node-gather-daemonset-524x8\" (UID: \"22cac32f-e82d-4e46-bbeb-09b8203e9e4e\") " pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-524x8" Sep 10 04:06:32.745146 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:32.744688 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/22cac32f-e82d-4e46-bbeb-09b8203e9e4e-podres\") pod \"perf-node-gather-daemonset-524x8\" (UID: \"22cac32f-e82d-4e46-bbeb-09b8203e9e4e\") " pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-524x8" Sep 10 04:06:32.745146 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:32.744710 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/22cac32f-e82d-4e46-bbeb-09b8203e9e4e-sys\") pod \"perf-node-gather-daemonset-524x8\" (UID: \"22cac32f-e82d-4e46-bbeb-09b8203e9e4e\") " pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-524x8" Sep 10 04:06:32.745146 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:32.744827 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/22cac32f-e82d-4e46-bbeb-09b8203e9e4e-proc\") pod \"perf-node-gather-daemonset-524x8\" (UID: \"22cac32f-e82d-4e46-bbeb-09b8203e9e4e\") " pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-524x8" Sep 10 04:06:32.845908 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:32.845866 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/22cac32f-e82d-4e46-bbeb-09b8203e9e4e-proc\") pod \"perf-node-gather-daemonset-524x8\" (UID: \"22cac32f-e82d-4e46-bbeb-09b8203e9e4e\") " pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-524x8" Sep 10 04:06:32.846113 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:32.845956 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vhbg\" (UniqueName: \"kubernetes.io/projected/22cac32f-e82d-4e46-bbeb-09b8203e9e4e-kube-api-access-5vhbg\") pod \"perf-node-gather-daemonset-524x8\" (UID: \"22cac32f-e82d-4e46-bbeb-09b8203e9e4e\") " pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-524x8" Sep 10 04:06:32.846113 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:32.845982 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/22cac32f-e82d-4e46-bbeb-09b8203e9e4e-lib-modules\") pod \"perf-node-gather-daemonset-524x8\" (UID: \"22cac32f-e82d-4e46-bbeb-09b8203e9e4e\") " pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-524x8" Sep 10 04:06:32.846113 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:32.845996 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/22cac32f-e82d-4e46-bbeb-09b8203e9e4e-proc\") pod \"perf-node-gather-daemonset-524x8\" (UID: \"22cac32f-e82d-4e46-bbeb-09b8203e9e4e\") " pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-524x8" Sep 10 04:06:32.846113 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:32.846016 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/22cac32f-e82d-4e46-bbeb-09b8203e9e4e-podres\") pod \"perf-node-gather-daemonset-524x8\" (UID: \"22cac32f-e82d-4e46-bbeb-09b8203e9e4e\") " pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-524x8" Sep 10 04:06:32.846113 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:32.846078 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/22cac32f-e82d-4e46-bbeb-09b8203e9e4e-sys\") pod \"perf-node-gather-daemonset-524x8\" (UID: \"22cac32f-e82d-4e46-bbeb-09b8203e9e4e\") " pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-524x8" Sep 10 04:06:32.846427 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:32.846134 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/22cac32f-e82d-4e46-bbeb-09b8203e9e4e-podres\") pod \"perf-node-gather-daemonset-524x8\" (UID: \"22cac32f-e82d-4e46-bbeb-09b8203e9e4e\") " pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-524x8" Sep 10 04:06:32.846427 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:32.846140 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/22cac32f-e82d-4e46-bbeb-09b8203e9e4e-lib-modules\") pod \"perf-node-gather-daemonset-524x8\" (UID: \"22cac32f-e82d-4e46-bbeb-09b8203e9e4e\") " pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-524x8" Sep 10 04:06:32.846427 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:32.846148 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/22cac32f-e82d-4e46-bbeb-09b8203e9e4e-sys\") pod \"perf-node-gather-daemonset-524x8\" (UID: \"22cac32f-e82d-4e46-bbeb-09b8203e9e4e\") " pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-524x8" Sep 10 04:06:32.854204 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:32.854174 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vhbg\" (UniqueName: \"kubernetes.io/projected/22cac32f-e82d-4e46-bbeb-09b8203e9e4e-kube-api-access-5vhbg\") pod \"perf-node-gather-daemonset-524x8\" (UID: \"22cac32f-e82d-4e46-bbeb-09b8203e9e4e\") " pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-524x8" Sep 10 04:06:32.968546 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:32.968445 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-524x8" Sep 10 04:06:33.237910 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:33.237818 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5qpb6/perf-node-gather-daemonset-524x8"] Sep 10 04:06:33.240870 ip-10-0-132-74 kubenswrapper[2562]: W0910 04:06:33.240837 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod22cac32f_e82d_4e46_bbeb_09b8203e9e4e.slice/crio-4b61d24a5d9987d4d00c8e6a2caba914e600a6f5e04bea3888c3ad970cbae78c WatchSource:0}: Error finding container 4b61d24a5d9987d4d00c8e6a2caba914e600a6f5e04bea3888c3ad970cbae78c: Status 404 returned error can't find the container with id 4b61d24a5d9987d4d00c8e6a2caba914e600a6f5e04bea3888c3ad970cbae78c Sep 10 04:06:33.376007 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:33.375971 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-524x8" event={"ID":"22cac32f-e82d-4e46-bbeb-09b8203e9e4e","Type":"ContainerStarted","Data":"9b3a73ec9cb1df64ac45f18931380ca974dab8c6e844cc810640a0a505775d08"} Sep 10 04:06:33.376195 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:33.376034 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-524x8" event={"ID":"22cac32f-e82d-4e46-bbeb-09b8203e9e4e","Type":"ContainerStarted","Data":"4b61d24a5d9987d4d00c8e6a2caba914e600a6f5e04bea3888c3ad970cbae78c"} Sep 10 04:06:33.376195 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:33.376075 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-524x8" Sep 10 04:06:33.391174 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:33.391120 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-524x8" podStartSLOduration=1.391104043 podStartE2EDuration="1.391104043s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 04:06:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:06:33.389613992 +0000 UTC m=+1120.265677146" watchObservedRunningTime="2026-09-10 04:06:33.391104043 +0000 UTC m=+1120.267167192" Sep 10 04:06:33.605937 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:33.605855 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-2zth2_881d05c0-8849-4caa-98fb-4a41ce400d67/volume-data-source-validator/0.log" Sep 10 04:06:34.660141 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:34.660106 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dkkkk_897fe719-a91a-4d75-8efb-4cb2fbade932/dns/0.log" Sep 10 04:06:34.680659 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:34.680629 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dkkkk_897fe719-a91a-4d75-8efb-4cb2fbade932/kube-rbac-proxy/0.log" Sep 10 04:06:34.843403 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:34.843372 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fwlqw_460613b7-2cdf-4e57-8edf-06e7b72f790a/dns-node-resolver/0.log" Sep 10 04:06:35.360518 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:35.360482 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6f7cc79777-ccvgm_17499bec-4013-4b5d-9d0c-008ee3db1541/registry/0.log" Sep 10 04:06:35.402621 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:35.402594 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nmf2n_1ef729ce-06d8-4c1d-ac8c-7a89770df17f/node-ca/0.log" Sep 10 04:06:36.430138 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:36.430103 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dcb5c9454-wstk2_41b98efd-b6d8-413c-9017-10e71dabaacf/kube-auth-proxy/0.log" Sep 10 04:06:36.473505 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:36.473464 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-hfppn_2d99147a-e8d3-4a02-b496-8d2cfeed1684/payload-pre-processing/0.log" Sep 10 04:06:37.221849 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:37.221811 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kx9jc_e6305575-8975-44bd-bd29-866c8457a56c/serve-healthcheck-canary/0.log" Sep 10 04:06:37.599703 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:37.599609 2562 scope.go:117] "RemoveContainer" containerID="606d636337a7c6642f4640fed2dfbeb5cd707a5d83ce0788f162dbb6a9567205" Sep 10 04:06:37.600164 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:06:37.599798 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f_llm(3c222592-85a8-4247-bd05-66be7a758411)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" podUID="3c222592-85a8-4247-bd05-66be7a758411" Sep 10 04:06:37.725528 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:37.725489 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-54dww_74b614da-e905-4ff7-9bfe-2d9bea24859b/kube-rbac-proxy/0.log" Sep 10 04:06:37.744633 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:37.744605 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-54dww_74b614da-e905-4ff7-9bfe-2d9bea24859b/exporter/0.log" Sep 10 04:06:37.766063 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:37.766023 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-54dww_74b614da-e905-4ff7-9bfe-2d9bea24859b/extractor/0.log" Sep 10 04:06:39.390512 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:39.390486 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-524x8" Sep 10 04:06:40.302879 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:40.302850 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-797557847b-nb65p_6a0f70b9-e517-4704-8019-12e9cfd9fd80/manager/0.log" Sep 10 04:06:40.320646 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:40.320618 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-797557847b-nb65p_6a0f70b9-e517-4704-8019-12e9cfd9fd80/copy-manifests/0.log" Sep 10 04:06:40.599774 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:40.599681 2562 scope.go:117] "RemoveContainer" containerID="77c2cb494206a72cfbd7ea653a7cbab8d8c7b05a3ba0b0c0cb2d90ecc8277bcd" Sep 10 04:06:40.600200 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:06:40.599890 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-v58d8_llm(638aee00-aa04-44c1-87a6-c1e179b7e82b)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-v58d8" podUID="638aee00-aa04-44c1-87a6-c1e179b7e82b" Sep 10 04:06:41.614773 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:41.614738 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-78c97bdccc-lrj57_43dacf9e-c15a-46f4-bd1d-1c114eb1a458/manager/0.log" Sep 10 04:06:43.370842 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:43.370803 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dcb5c9454-wstk2_41b98efd-b6d8-413c-9017-10e71dabaacf/kube-auth-proxy/0.log" Sep 10 04:06:43.402130 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:43.402096 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-hfppn_2d99147a-e8d3-4a02-b496-8d2cfeed1684/payload-pre-processing/0.log" Sep 10 04:06:44.622180 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:44.622141 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dcb5c9454-wstk2_41b98efd-b6d8-413c-9017-10e71dabaacf/kube-auth-proxy/0.log" Sep 10 04:06:44.653392 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:44.653309 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-hfppn_2d99147a-e8d3-4a02-b496-8d2cfeed1684/payload-pre-processing/0.log" Sep 10 04:06:46.804590 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:46.804556 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-gz76m_4f351318-0f05-4b9c-9266-73b5ca137fa8/migrator/0.log" Sep 10 04:06:46.823230 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:46.823193 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-gz76m_4f351318-0f05-4b9c-9266-73b5ca137fa8/graceful-termination/0.log" Sep 10 04:06:48.323468 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:48.323438 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ckqrf_aae7f13d-be0c-4bd9-af61-40fc05f144c0/kube-multus-additional-cni-plugins/0.log" Sep 10 04:06:48.344327 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:48.344282 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ckqrf_aae7f13d-be0c-4bd9-af61-40fc05f144c0/egress-router-binary-copy/0.log" Sep 10 04:06:48.370148 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:48.370092 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ckqrf_aae7f13d-be0c-4bd9-af61-40fc05f144c0/cni-plugins/0.log" Sep 10 04:06:48.389248 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:48.389213 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ckqrf_aae7f13d-be0c-4bd9-af61-40fc05f144c0/bond-cni-plugin/0.log" Sep 10 04:06:48.407530 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:48.407495 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ckqrf_aae7f13d-be0c-4bd9-af61-40fc05f144c0/routeoverride-cni/0.log" Sep 10 04:06:48.425800 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:48.425765 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ckqrf_aae7f13d-be0c-4bd9-af61-40fc05f144c0/whereabouts-cni-bincopy/0.log" Sep 10 04:06:48.443656 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:48.443624 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ckqrf_aae7f13d-be0c-4bd9-af61-40fc05f144c0/whereabouts-cni/0.log" Sep 10 04:06:48.598856 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:48.598773 2562 scope.go:117] "RemoveContainer" containerID="606d636337a7c6642f4640fed2dfbeb5cd707a5d83ce0788f162dbb6a9567205" Sep 10 04:06:48.599008 ip-10-0-132-74 kubenswrapper[2562]: E0910 04:06:48.598971 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f_llm(3c222592-85a8-4247-bd05-66be7a758411)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-zbq9f" podUID="3c222592-85a8-4247-bd05-66be7a758411" Sep 10 04:06:48.665835 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:48.665800 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-vwrxj_b0936f72-795e-4e46-8316-ecc03bb2dec6/kube-multus/0.log" Sep 10 04:06:48.717693 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:48.717655 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-2p9b8_158bac16-c713-47cd-91d7-9fc36d96f570/network-metrics-daemon/0.log" Sep 10 04:06:48.742524 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:48.742488 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-2p9b8_158bac16-c713-47cd-91d7-9fc36d96f570/kube-rbac-proxy/0.log" Sep 10 04:06:49.936840 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:49.936787 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fl7hj_8459a996-8d64-4c23-9e92-1a5bcca253b5/ovn-controller/0.log" Sep 10 04:06:49.951825 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:49.951796 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fl7hj_8459a996-8d64-4c23-9e92-1a5bcca253b5/ovn-acl-logging/0.log" Sep 10 04:06:49.952000 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:49.951964 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fl7hj_8459a996-8d64-4c23-9e92-1a5bcca253b5/ovn-acl-logging/1.log" Sep 10 04:06:49.977232 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:49.977184 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fl7hj_8459a996-8d64-4c23-9e92-1a5bcca253b5/kube-rbac-proxy-node/0.log" Sep 10 04:06:50.002590 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:50.002560 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fl7hj_8459a996-8d64-4c23-9e92-1a5bcca253b5/kube-rbac-proxy-ovn-metrics/0.log" Sep 10 04:06:50.018060 ip-10-0-132-74 kubenswrapper[2562]: I0910 04:06:50.018024 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fl7hj_8459a996-8d64-4c23-9e92-1a5bcca253b5/northd/0.log"