Sep 10 02:49:01.032819 ip-10-0-133-37 systemd[1]: Starting Kubernetes Kubelet... Sep 10 02:49:01.425076 ip-10-0-133-37 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 02:49:01.425076 ip-10-0-133-37 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 10 02:49:01.425076 ip-10-0-133-37 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 02:49:01.425076 ip-10-0-133-37 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 10 02:49:01.425076 ip-10-0-133-37 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 02:49:01.425936 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.425846 2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 10 02:49:01.429614 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429598 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 02:49:01.429614 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429614 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 02:49:01.429682 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429618 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 02:49:01.429682 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429621 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 02:49:01.429682 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429624 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 02:49:01.429682 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429627 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 02:49:01.429682 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429630 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 02:49:01.429682 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429633 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 02:49:01.429682 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429636 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 02:49:01.429682 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429639 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 02:49:01.429682 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429642 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 02:49:01.429682 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429645 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 02:49:01.429682 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429647 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 02:49:01.429682 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429650 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 02:49:01.429682 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429653 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 02:49:01.429682 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429658 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 02:49:01.429682 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429663 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 02:49:01.429682 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429667 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 02:49:01.429682 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429670 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 02:49:01.429682 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429676 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 02:49:01.429682 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429679 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 02:49:01.430128 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429685 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 02:49:01.430128 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429688 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 02:49:01.430128 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429692 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 02:49:01.430128 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429695 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 02:49:01.430128 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429698 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 02:49:01.430128 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429701 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 02:49:01.430128 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429704 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 02:49:01.430128 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429707 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 02:49:01.430128 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429709 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 02:49:01.430128 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429712 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 02:49:01.430128 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429715 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 02:49:01.430128 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429718 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 02:49:01.430128 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429720 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 02:49:01.430128 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429723 2560 feature_gate.go:328] unrecognized feature gate: Example Sep 10 02:49:01.430128 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429726 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 02:49:01.430128 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429728 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 02:49:01.430128 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429731 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 02:49:01.430128 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429734 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 02:49:01.430128 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429736 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 02:49:01.430128 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429739 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 02:49:01.430636 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429741 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 02:49:01.430636 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429743 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 02:49:01.430636 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429746 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 02:49:01.430636 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429748 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 02:49:01.430636 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429751 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 02:49:01.430636 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429753 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 02:49:01.430636 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429756 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 02:49:01.430636 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429758 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 02:49:01.430636 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429760 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 02:49:01.430636 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429764 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 02:49:01.430636 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429767 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 02:49:01.430636 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429769 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 02:49:01.430636 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429771 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 02:49:01.430636 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429775 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 02:49:01.430636 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429778 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 02:49:01.430636 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429780 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 02:49:01.430636 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429783 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 02:49:01.430636 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429785 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 02:49:01.430636 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429788 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 02:49:01.431091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429790 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 02:49:01.431091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429793 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 02:49:01.431091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429795 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 02:49:01.431091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429798 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 02:49:01.431091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429800 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 02:49:01.431091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429802 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 02:49:01.431091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429805 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 02:49:01.431091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429807 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 02:49:01.431091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429810 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 02:49:01.431091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429812 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 02:49:01.431091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429814 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 02:49:01.431091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429818 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 02:49:01.431091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429820 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 02:49:01.431091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429823 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 02:49:01.431091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429825 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 02:49:01.431091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429827 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 02:49:01.431091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429830 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 02:49:01.431091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429833 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 02:49:01.431091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429837 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 02:49:01.431091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429839 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 02:49:01.431580 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429842 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 02:49:01.431580 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429844 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 02:49:01.431580 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429847 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 02:49:01.431580 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429849 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 02:49:01.431580 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429852 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 02:49:01.431580 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429854 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 02:49:01.431580 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429856 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 02:49:01.431580 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429859 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 02:49:01.431580 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429862 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 02:49:01.431580 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429864 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 02:49:01.431580 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429866 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 02:49:01.431580 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429869 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 02:49:01.431580 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429871 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 02:49:01.431580 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429873 2560 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 02:49:01.431580 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429876 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 02:49:01.431580 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429878 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 02:49:01.431580 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429880 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 02:49:01.431580 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429883 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 02:49:01.431580 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.429885 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 02:49:01.431580 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430605 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 02:49:01.432091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430614 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 02:49:01.432091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430618 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 02:49:01.432091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430621 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 02:49:01.432091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430624 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 02:49:01.432091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430627 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 02:49:01.432091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430631 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 02:49:01.432091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430635 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 02:49:01.432091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430638 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 02:49:01.432091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430641 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 02:49:01.432091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430645 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 02:49:01.432091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430648 2560 feature_gate.go:328] unrecognized feature gate: Example Sep 10 02:49:01.432091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430651 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 02:49:01.432091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430653 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 02:49:01.432091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430656 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 02:49:01.432091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430659 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 02:49:01.432091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430661 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 02:49:01.432091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430664 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 02:49:01.432091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430666 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 02:49:01.432091 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430671 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 02:49:01.432588 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430674 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 02:49:01.432588 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430676 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 02:49:01.432588 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430679 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 02:49:01.432588 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430681 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 02:49:01.432588 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430684 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 02:49:01.432588 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430686 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 02:49:01.432588 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430689 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 02:49:01.432588 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430692 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 02:49:01.432588 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430695 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 02:49:01.432588 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430697 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 02:49:01.432588 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430700 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 02:49:01.432588 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430702 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 02:49:01.432588 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430704 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 02:49:01.432588 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430707 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 02:49:01.432588 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430709 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 02:49:01.432588 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430713 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 02:49:01.432588 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430716 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 02:49:01.432588 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430718 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 02:49:01.432588 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430726 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 02:49:01.432588 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430729 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 02:49:01.433078 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430731 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 02:49:01.433078 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430734 2560 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 02:49:01.433078 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430736 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 02:49:01.433078 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430739 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 02:49:01.433078 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430742 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 02:49:01.433078 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430744 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 02:49:01.433078 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430747 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 02:49:01.433078 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430749 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 02:49:01.433078 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430752 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 02:49:01.433078 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430754 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 02:49:01.433078 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430757 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 02:49:01.433078 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430761 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 02:49:01.433078 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430764 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 02:49:01.433078 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430767 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 02:49:01.433078 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430769 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 02:49:01.433078 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430772 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 02:49:01.433078 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430774 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 02:49:01.433078 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430777 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 02:49:01.433078 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430779 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 02:49:01.433078 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430782 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 02:49:01.433568 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430785 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 02:49:01.433568 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430787 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 02:49:01.433568 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430789 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 02:49:01.433568 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430792 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 02:49:01.433568 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430795 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 02:49:01.433568 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430797 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 02:49:01.433568 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430800 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 02:49:01.433568 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430802 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 02:49:01.433568 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430804 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 02:49:01.433568 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430808 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 02:49:01.433568 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430810 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 02:49:01.433568 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430818 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 02:49:01.433568 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430820 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 02:49:01.433568 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430823 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 02:49:01.433568 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430825 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 02:49:01.433568 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430828 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 02:49:01.433568 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430831 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 02:49:01.433568 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430833 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 02:49:01.433568 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430836 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 02:49:01.433568 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430839 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 02:49:01.434051 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430841 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 02:49:01.434051 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430844 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 02:49:01.434051 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430846 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 02:49:01.434051 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430850 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 02:49:01.434051 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430853 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 02:49:01.434051 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430855 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 02:49:01.434051 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430858 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 02:49:01.434051 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430862 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 02:49:01.434051 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430865 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 02:49:01.434051 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430868 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 02:49:01.434051 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430871 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 02:49:01.434051 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430873 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 02:49:01.434051 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430875 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 02:49:01.434051 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430878 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 02:49:01.434051 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430881 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 02:49:01.434051 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430883 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 02:49:01.434051 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430886 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 02:49:01.434051 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430888 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 02:49:01.434051 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.430891 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 02:49:01.434051 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431298 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 02:49:01.434554 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431303 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 02:49:01.434554 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431306 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 02:49:01.434554 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431309 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 02:49:01.434554 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431313 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 02:49:01.434554 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431315 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 02:49:01.434554 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431318 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 02:49:01.434554 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431322 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 02:49:01.434554 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431326 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 02:49:01.434554 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431329 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 02:49:01.434554 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431332 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 02:49:01.434554 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431334 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 02:49:01.434554 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431337 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 02:49:01.434554 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431339 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 02:49:01.434554 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431341 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 02:49:01.434554 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431344 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 02:49:01.434554 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431348 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 02:49:01.434554 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431350 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 02:49:01.434554 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431353 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 02:49:01.434554 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431355 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 02:49:01.434554 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431358 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 02:49:01.435041 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431360 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 02:49:01.435041 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431363 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 02:49:01.435041 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431365 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 02:49:01.435041 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431368 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 02:49:01.435041 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431370 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 02:49:01.435041 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431373 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 02:49:01.435041 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431376 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 02:49:01.435041 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431378 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 02:49:01.435041 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431382 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 02:49:01.435041 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431385 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 02:49:01.435041 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431387 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 02:49:01.435041 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431391 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 02:49:01.435041 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431393 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 02:49:01.435041 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431396 2560 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 02:49:01.435041 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431399 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 02:49:01.435041 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431401 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 02:49:01.435041 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431404 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 02:49:01.435041 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431407 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 02:49:01.435041 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431410 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 02:49:01.435495 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431413 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 02:49:01.435495 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431416 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 02:49:01.435495 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431418 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 02:49:01.435495 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431421 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 02:49:01.435495 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431424 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 02:49:01.435495 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431426 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 02:49:01.435495 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431429 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 02:49:01.435495 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431431 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 02:49:01.435495 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431435 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 02:49:01.435495 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431438 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 02:49:01.435495 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431440 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 02:49:01.435495 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431442 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 02:49:01.435495 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431445 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 02:49:01.435495 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431447 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 02:49:01.435495 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431450 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 02:49:01.435495 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431453 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 02:49:01.435495 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431455 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 02:49:01.435495 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431458 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 02:49:01.435495 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431460 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 02:49:01.435495 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431463 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 02:49:01.435994 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431465 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 02:49:01.435994 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431469 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 02:49:01.435994 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431471 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 02:49:01.435994 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431473 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 02:49:01.435994 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431476 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 02:49:01.435994 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431479 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 02:49:01.435994 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431481 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 02:49:01.435994 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431483 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 02:49:01.435994 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431486 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 02:49:01.435994 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431488 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 02:49:01.435994 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431491 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 02:49:01.435994 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431493 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 02:49:01.435994 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431496 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 02:49:01.435994 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431499 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 02:49:01.435994 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431501 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 02:49:01.435994 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431520 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 02:49:01.435994 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431523 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 02:49:01.435994 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431525 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 02:49:01.435994 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431528 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 02:49:01.435994 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431532 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 02:49:01.436524 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431536 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 02:49:01.436524 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431539 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 02:49:01.436524 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431542 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 02:49:01.436524 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431545 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 02:49:01.436524 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431547 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 02:49:01.436524 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431550 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 02:49:01.436524 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431553 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 02:49:01.436524 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431555 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 02:49:01.436524 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431558 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 02:49:01.436524 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431560 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 02:49:01.436524 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431563 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 02:49:01.436524 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431565 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 02:49:01.436524 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431568 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 02:49:01.436524 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431570 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 02:49:01.436524 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431572 2560 feature_gate.go:328] unrecognized feature gate: Example Sep 10 02:49:01.436524 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431575 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 02:49:01.436524 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431578 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 02:49:01.436524 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431580 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 02:49:01.436524 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.431583 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 02:49:01.436524 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431654 2560 flags.go:64] FLAG: --address="0.0.0.0" Sep 10 02:49:01.437028 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431687 2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 10 02:49:01.437028 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431694 2560 flags.go:64] FLAG: --anonymous-auth="true" Sep 10 02:49:01.437028 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431699 2560 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 10 02:49:01.437028 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431704 2560 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 10 02:49:01.437028 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431707 2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 10 02:49:01.437028 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431712 2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 10 02:49:01.437028 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431716 2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 10 02:49:01.437028 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431720 2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 10 02:49:01.437028 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431722 2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 10 02:49:01.437028 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431726 2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 10 02:49:01.437028 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431730 2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 10 02:49:01.437028 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431733 2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 10 02:49:01.437028 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431737 2560 flags.go:64] FLAG: --cgroup-root="" Sep 10 02:49:01.437028 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431740 2560 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 10 02:49:01.437028 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431743 2560 flags.go:64] FLAG: --client-ca-file="" Sep 10 02:49:01.437028 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431746 2560 flags.go:64] FLAG: --cloud-provider="external" Sep 10 02:49:01.437028 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431749 2560 flags.go:64] FLAG: --cluster-dns="[]" Sep 10 02:49:01.437028 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431753 2560 flags.go:64] FLAG: --cluster-domain="" Sep 10 02:49:01.437028 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431755 2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 10 02:49:01.437028 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431759 2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 10 02:49:01.437028 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431762 2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 10 02:49:01.437028 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431765 2560 flags.go:64] FLAG: --container-log-max-files="5" Sep 10 02:49:01.437028 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431769 2560 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 10 02:49:01.437028 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431772 2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 10 02:49:01.437620 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431775 2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 10 02:49:01.437620 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431779 2560 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 10 02:49:01.437620 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431782 2560 flags.go:64] FLAG: --contention-profiling="false" Sep 10 02:49:01.437620 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431784 2560 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 10 02:49:01.437620 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431787 2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 10 02:49:01.437620 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431790 2560 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 10 02:49:01.437620 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431793 2560 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 10 02:49:01.437620 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431797 2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 10 02:49:01.437620 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431800 2560 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 10 02:49:01.437620 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431803 2560 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 10 02:49:01.437620 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431805 2560 flags.go:64] FLAG: --enable-load-reader="false" Sep 10 02:49:01.437620 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431808 2560 flags.go:64] FLAG: --enable-server="true" Sep 10 02:49:01.437620 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431811 2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 10 02:49:01.437620 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431816 2560 flags.go:64] FLAG: --event-burst="100" Sep 10 02:49:01.437620 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431819 2560 flags.go:64] FLAG: --event-qps="50" Sep 10 02:49:01.437620 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431822 2560 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 10 02:49:01.437620 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431825 2560 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 10 02:49:01.437620 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431828 2560 flags.go:64] FLAG: --eviction-hard="" Sep 10 02:49:01.437620 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431833 2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 10 02:49:01.437620 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431835 2560 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 10 02:49:01.437620 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431839 2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 10 02:49:01.437620 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431842 2560 flags.go:64] FLAG: --eviction-soft="" Sep 10 02:49:01.437620 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431845 2560 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 10 02:49:01.437620 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431848 2560 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 10 02:49:01.437620 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431851 2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 10 02:49:01.438330 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431853 2560 flags.go:64] FLAG: --experimental-mounter-path="" Sep 10 02:49:01.438330 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431856 2560 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 10 02:49:01.438330 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431859 2560 flags.go:64] FLAG: --fail-swap-on="true" Sep 10 02:49:01.438330 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431862 2560 flags.go:64] FLAG: --feature-gates="" Sep 10 02:49:01.438330 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431866 2560 flags.go:64] FLAG: --file-check-frequency="20s" Sep 10 02:49:01.438330 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431869 2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 10 02:49:01.438330 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431872 2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 10 02:49:01.438330 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431875 2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 10 02:49:01.438330 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431878 2560 flags.go:64] FLAG: --healthz-port="10248" Sep 10 02:49:01.438330 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431880 2560 flags.go:64] FLAG: --help="false" Sep 10 02:49:01.438330 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431884 2560 flags.go:64] FLAG: --hostname-override="ip-10-0-133-37.ec2.internal" Sep 10 02:49:01.438330 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431887 2560 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 10 02:49:01.438330 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431889 2560 flags.go:64] FLAG: --http-check-frequency="20s" Sep 10 02:49:01.438330 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431892 2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 10 02:49:01.438330 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431895 2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 10 02:49:01.438330 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431899 2560 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 10 02:49:01.438330 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431902 2560 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 10 02:49:01.438330 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431905 2560 flags.go:64] FLAG: --image-service-endpoint="" Sep 10 02:49:01.438330 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431908 2560 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 10 02:49:01.438330 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431910 2560 flags.go:64] FLAG: --kube-api-burst="100" Sep 10 02:49:01.438330 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431914 2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 10 02:49:01.438330 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431917 2560 flags.go:64] FLAG: --kube-api-qps="50" Sep 10 02:49:01.438330 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431920 2560 flags.go:64] FLAG: --kube-reserved="" Sep 10 02:49:01.438330 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431923 2560 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 10 02:49:01.438950 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431925 2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 10 02:49:01.438950 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431930 2560 flags.go:64] FLAG: --kubelet-cgroups="" Sep 10 02:49:01.438950 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431932 2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 10 02:49:01.438950 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431935 2560 flags.go:64] FLAG: --lock-file="" Sep 10 02:49:01.438950 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431938 2560 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 10 02:49:01.438950 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431941 2560 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 10 02:49:01.438950 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431944 2560 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 10 02:49:01.438950 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431950 2560 flags.go:64] FLAG: --log-json-split-stream="false" Sep 10 02:49:01.438950 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431952 2560 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 10 02:49:01.438950 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431955 2560 flags.go:64] FLAG: --log-text-split-stream="false" Sep 10 02:49:01.438950 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431958 2560 flags.go:64] FLAG: --logging-format="text" Sep 10 02:49:01.438950 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431961 2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 10 02:49:01.438950 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431964 2560 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 10 02:49:01.438950 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431967 2560 flags.go:64] FLAG: --manifest-url="" Sep 10 02:49:01.438950 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431970 2560 flags.go:64] FLAG: --manifest-url-header="" Sep 10 02:49:01.438950 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431974 2560 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 10 02:49:01.438950 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431977 2560 flags.go:64] FLAG: --max-open-files="1000000" Sep 10 02:49:01.438950 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431981 2560 flags.go:64] FLAG: --max-pods="110" Sep 10 02:49:01.438950 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431984 2560 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 10 02:49:01.438950 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431987 2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 10 02:49:01.438950 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431990 2560 flags.go:64] FLAG: --memory-manager-policy="None" Sep 10 02:49:01.438950 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431993 2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 10 02:49:01.438950 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431996 2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 10 02:49:01.438950 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.431999 2560 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 10 02:49:01.438950 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432002 2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 10 02:49:01.439577 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432010 2560 flags.go:64] FLAG: --node-status-max-images="50" Sep 10 02:49:01.439577 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432012 2560 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 10 02:49:01.439577 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432015 2560 flags.go:64] FLAG: --oom-score-adj="-999" Sep 10 02:49:01.439577 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432018 2560 flags.go:64] FLAG: --pod-cidr="" Sep 10 02:49:01.439577 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432022 2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 10 02:49:01.439577 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432033 2560 flags.go:64] FLAG: --pod-manifest-path="" Sep 10 02:49:01.439577 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432036 2560 flags.go:64] FLAG: --pod-max-pids="-1" Sep 10 02:49:01.439577 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432039 2560 flags.go:64] FLAG: --pods-per-core="0" Sep 10 02:49:01.439577 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432043 2560 flags.go:64] FLAG: --port="10250" Sep 10 02:49:01.439577 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432046 2560 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 10 02:49:01.439577 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432049 2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-05541d6293b3c79b7" Sep 10 02:49:01.439577 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432053 2560 flags.go:64] FLAG: --qos-reserved="" Sep 10 02:49:01.439577 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432055 2560 flags.go:64] FLAG: --read-only-port="10255" Sep 10 02:49:01.439577 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432058 2560 flags.go:64] FLAG: --register-node="true" Sep 10 02:49:01.439577 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432061 2560 flags.go:64] FLAG: --register-with-taints="" Sep 10 02:49:01.439577 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432069 2560 flags.go:64] FLAG: --registry-burst="10" Sep 10 02:49:01.439577 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432071 2560 flags.go:64] FLAG: --registry-qps="5" Sep 10 02:49:01.439577 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432074 2560 flags.go:64] FLAG: --reserved-cpus="" Sep 10 02:49:01.439577 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432077 2560 flags.go:64] FLAG: --reserved-memory="" Sep 10 02:49:01.439577 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432080 2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 10 02:49:01.439577 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432083 2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 10 02:49:01.439577 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432086 2560 flags.go:64] FLAG: --rotate-certificates="false" Sep 10 02:49:01.439577 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432088 2560 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 10 02:49:01.439577 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432092 2560 flags.go:64] FLAG: --runonce="false" Sep 10 02:49:01.439577 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432095 2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 10 02:49:01.440212 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432098 2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 10 02:49:01.440212 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432101 2560 flags.go:64] FLAG: --seccomp-default="false" Sep 10 02:49:01.440212 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432104 2560 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 10 02:49:01.440212 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432106 2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 10 02:49:01.440212 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432109 2560 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 10 02:49:01.440212 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432112 2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 10 02:49:01.440212 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432115 2560 flags.go:64] FLAG: --storage-driver-password="root" Sep 10 02:49:01.440212 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432118 2560 flags.go:64] FLAG: --storage-driver-secure="false" Sep 10 02:49:01.440212 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432120 2560 flags.go:64] FLAG: --storage-driver-table="stats" Sep 10 02:49:01.440212 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432123 2560 flags.go:64] FLAG: --storage-driver-user="root" Sep 10 02:49:01.440212 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432126 2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 10 02:49:01.440212 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432129 2560 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 10 02:49:01.440212 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432134 2560 flags.go:64] FLAG: --system-cgroups="" Sep 10 02:49:01.440212 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432136 2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 10 02:49:01.440212 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432142 2560 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 10 02:49:01.440212 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432145 2560 flags.go:64] FLAG: --tls-cert-file="" Sep 10 02:49:01.440212 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432148 2560 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 10 02:49:01.440212 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432152 2560 flags.go:64] FLAG: --tls-min-version="" Sep 10 02:49:01.440212 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432156 2560 flags.go:64] FLAG: --tls-private-key-file="" Sep 10 02:49:01.440212 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432159 2560 flags.go:64] FLAG: --topology-manager-policy="none" Sep 10 02:49:01.440212 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432162 2560 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 10 02:49:01.440212 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432164 2560 flags.go:64] FLAG: --topology-manager-scope="container" Sep 10 02:49:01.440212 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432167 2560 flags.go:64] FLAG: --v="2" Sep 10 02:49:01.440212 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432172 2560 flags.go:64] FLAG: --version="false" Sep 10 02:49:01.440212 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432176 2560 flags.go:64] FLAG: --vmodule="" Sep 10 02:49:01.440853 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432179 2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 10 02:49:01.440853 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.432182 2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 10 02:49:01.440853 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432301 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 02:49:01.440853 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432304 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 02:49:01.440853 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432307 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 02:49:01.440853 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432310 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 02:49:01.440853 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432313 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 02:49:01.440853 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432316 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 02:49:01.440853 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432318 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 02:49:01.440853 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432321 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 02:49:01.440853 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432323 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 02:49:01.440853 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432326 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 02:49:01.440853 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432328 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 02:49:01.440853 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432331 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 02:49:01.440853 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432334 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 02:49:01.440853 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432336 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 02:49:01.440853 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432339 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 02:49:01.440853 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432341 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 02:49:01.440853 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432344 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 02:49:01.440853 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432346 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 02:49:01.441353 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432349 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 02:49:01.441353 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432352 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 02:49:01.441353 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432357 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 02:49:01.441353 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432361 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 02:49:01.441353 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432364 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 02:49:01.441353 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432368 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 02:49:01.441353 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432371 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 02:49:01.441353 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432374 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 02:49:01.441353 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432376 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 02:49:01.441353 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432379 2560 feature_gate.go:328] unrecognized feature gate: Example Sep 10 02:49:01.441353 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432381 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 02:49:01.441353 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432384 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 02:49:01.441353 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432386 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 02:49:01.441353 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432389 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 02:49:01.441353 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432392 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 02:49:01.441353 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432394 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 02:49:01.441353 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432397 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 02:49:01.441353 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432399 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 02:49:01.441353 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432402 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 02:49:01.441851 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432404 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 02:49:01.441851 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432406 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 02:49:01.441851 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432409 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 02:49:01.441851 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432411 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 02:49:01.441851 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432414 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 02:49:01.441851 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432416 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 02:49:01.441851 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432418 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 02:49:01.441851 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432421 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 02:49:01.441851 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432423 2560 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 02:49:01.441851 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432426 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 02:49:01.441851 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432428 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 02:49:01.441851 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432430 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 02:49:01.441851 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432433 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 02:49:01.441851 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432435 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 02:49:01.441851 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432438 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 02:49:01.441851 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432443 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 02:49:01.441851 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432445 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 02:49:01.441851 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432447 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 02:49:01.441851 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432451 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 02:49:01.441851 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432453 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 02:49:01.442340 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432456 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 02:49:01.442340 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432460 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 02:49:01.442340 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432463 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 02:49:01.442340 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432466 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 02:49:01.442340 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432468 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 02:49:01.442340 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432471 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 02:49:01.442340 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432474 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 02:49:01.442340 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432476 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 02:49:01.442340 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432479 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 02:49:01.442340 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432481 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 02:49:01.442340 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432484 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 02:49:01.442340 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432486 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 02:49:01.442340 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432489 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 02:49:01.442340 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432492 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 02:49:01.442340 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432496 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 02:49:01.442340 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432499 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 02:49:01.442340 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432501 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 02:49:01.442340 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432520 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 02:49:01.442340 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432523 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 02:49:01.442873 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432528 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 02:49:01.442873 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432530 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 02:49:01.442873 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432533 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 02:49:01.442873 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432535 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 02:49:01.442873 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432538 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 02:49:01.442873 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432548 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 02:49:01.442873 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432551 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 02:49:01.442873 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432553 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 02:49:01.442873 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432559 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 02:49:01.442873 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432561 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 02:49:01.442873 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432564 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 02:49:01.442873 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432567 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 02:49:01.442873 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432570 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 02:49:01.442873 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432572 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 02:49:01.442873 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432575 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 02:49:01.442873 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432577 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 02:49:01.442873 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432580 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 02:49:01.442873 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432582 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 02:49:01.442873 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432585 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 02:49:01.442873 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432587 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 02:49:01.443368 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432590 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 02:49:01.443368 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432593 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 02:49:01.443368 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.432596 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 02:49:01.443368 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.433153 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 02:49:01.443368 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.440419 2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 10 02:49:01.443368 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.440438 2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 10 02:49:01.443368 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.441122 2560 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 10 02:49:01.443368 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.441130 2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 10 02:49:01.443368 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441194 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 02:49:01.443368 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441200 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 02:49:01.443368 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441204 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 02:49:01.443368 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441207 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 02:49:01.443368 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441209 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 02:49:01.443368 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441215 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 02:49:01.443368 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441219 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 02:49:01.443802 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441222 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 02:49:01.443802 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441225 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 02:49:01.443802 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441228 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 02:49:01.443802 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441230 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 02:49:01.443802 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441233 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 02:49:01.443802 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441235 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 02:49:01.443802 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441238 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 02:49:01.443802 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441241 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 02:49:01.443802 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441243 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 02:49:01.443802 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441247 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 02:49:01.443802 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441249 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 02:49:01.443802 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441254 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 02:49:01.443802 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441258 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 02:49:01.443802 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441261 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 02:49:01.443802 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441264 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 02:49:01.443802 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441266 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 02:49:01.443802 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441269 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 02:49:01.443802 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441272 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 02:49:01.443802 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441274 2560 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 02:49:01.443802 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441277 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 02:49:01.444295 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441280 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 02:49:01.444295 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441282 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 02:49:01.444295 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441286 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 02:49:01.444295 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441289 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 02:49:01.444295 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441291 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 02:49:01.444295 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441294 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 02:49:01.444295 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441296 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 02:49:01.444295 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441299 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 02:49:01.444295 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441302 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 02:49:01.444295 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441304 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 02:49:01.444295 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441307 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 02:49:01.444295 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441309 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 02:49:01.444295 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441312 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 02:49:01.444295 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441314 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 02:49:01.444295 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441317 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 02:49:01.444295 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441321 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 02:49:01.444295 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441324 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 02:49:01.444295 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441326 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 02:49:01.444295 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441329 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 02:49:01.444777 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441332 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 02:49:01.444777 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441335 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 02:49:01.444777 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441337 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 02:49:01.444777 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441340 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 02:49:01.444777 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441343 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 02:49:01.444777 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441345 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 02:49:01.444777 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441347 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 02:49:01.444777 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441351 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 02:49:01.444777 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441354 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 02:49:01.444777 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441357 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 02:49:01.444777 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441359 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 02:49:01.444777 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441362 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 02:49:01.444777 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441365 2560 feature_gate.go:328] unrecognized feature gate: Example Sep 10 02:49:01.444777 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441368 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 02:49:01.444777 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441370 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 02:49:01.444777 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441373 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 02:49:01.444777 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441376 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 02:49:01.444777 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441379 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 02:49:01.444777 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441382 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 02:49:01.444777 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441385 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 02:49:01.445270 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441387 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 02:49:01.445270 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441390 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 02:49:01.445270 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441393 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 02:49:01.445270 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441395 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 02:49:01.445270 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441398 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 02:49:01.445270 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441400 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 02:49:01.445270 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441403 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 02:49:01.445270 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441405 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 02:49:01.445270 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441408 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 02:49:01.445270 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441411 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 02:49:01.445270 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441413 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 02:49:01.445270 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441415 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 02:49:01.445270 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441418 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 02:49:01.445270 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441421 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 02:49:01.445270 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441423 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 02:49:01.445270 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441426 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 02:49:01.445270 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441428 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 02:49:01.445270 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441431 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 02:49:01.445270 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441434 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 02:49:01.445270 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441437 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 02:49:01.445772 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441439 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 02:49:01.445772 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441442 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 02:49:01.445772 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441444 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 02:49:01.445772 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441447 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 02:49:01.445772 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441449 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 02:49:01.445772 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441453 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 02:49:01.445772 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441455 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 02:49:01.445772 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441458 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 02:49:01.445772 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441460 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 02:49:01.445772 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441463 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 02:49:01.445772 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441466 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 02:49:01.445772 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441468 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 02:49:01.445772 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441471 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 02:49:01.445772 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.441476 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 02:49:01.445772 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441615 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 02:49:01.446145 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441621 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 02:49:01.446145 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441624 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 02:49:01.446145 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441627 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 02:49:01.446145 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441631 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 02:49:01.446145 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441636 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 02:49:01.446145 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441639 2560 feature_gate.go:328] unrecognized feature gate: Example Sep 10 02:49:01.446145 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441641 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 02:49:01.446145 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441644 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 02:49:01.446145 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441647 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 02:49:01.446145 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441650 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 02:49:01.446145 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441652 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 02:49:01.446145 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441655 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 02:49:01.446145 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441657 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 02:49:01.446145 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441660 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 02:49:01.446145 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441662 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 02:49:01.446145 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441665 2560 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 02:49:01.446145 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441668 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 02:49:01.446145 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441671 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 02:49:01.446145 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441673 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 02:49:01.446145 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441676 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 02:49:01.446709 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441678 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 02:49:01.446709 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441681 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 02:49:01.446709 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441683 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 02:49:01.446709 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441686 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 02:49:01.446709 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441689 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 02:49:01.446709 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441692 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 02:49:01.446709 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441694 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 02:49:01.446709 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441697 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 02:49:01.446709 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441700 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 02:49:01.446709 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441702 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 02:49:01.446709 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441705 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 02:49:01.446709 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441707 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 02:49:01.446709 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441711 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 02:49:01.446709 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441714 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 02:49:01.446709 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441717 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 02:49:01.446709 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441720 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 02:49:01.446709 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441723 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 02:49:01.446709 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441726 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 02:49:01.446709 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441728 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 02:49:01.446709 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441731 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 02:49:01.447183 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441734 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 02:49:01.447183 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441737 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 02:49:01.447183 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441739 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 02:49:01.447183 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441742 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 02:49:01.447183 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441745 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 02:49:01.447183 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441747 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 02:49:01.447183 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441750 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 02:49:01.447183 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441753 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 02:49:01.447183 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441755 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 02:49:01.447183 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441758 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 02:49:01.447183 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441760 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 02:49:01.447183 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441763 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 02:49:01.447183 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441765 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 02:49:01.447183 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441768 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 02:49:01.447183 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441770 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 02:49:01.447183 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441773 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 02:49:01.447183 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441775 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 02:49:01.447183 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441778 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 02:49:01.447183 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441781 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 02:49:01.447183 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441783 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 02:49:01.447684 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441787 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 02:49:01.447684 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441789 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 02:49:01.447684 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441792 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 02:49:01.447684 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441795 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 02:49:01.447684 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441797 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 02:49:01.447684 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441799 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 02:49:01.447684 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441802 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 02:49:01.447684 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441805 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 02:49:01.447684 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441807 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 02:49:01.447684 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441810 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 02:49:01.447684 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441812 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 02:49:01.447684 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441815 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 02:49:01.447684 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441817 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 02:49:01.447684 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441819 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 02:49:01.447684 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441822 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 02:49:01.447684 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441824 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 02:49:01.447684 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441827 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 02:49:01.447684 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441829 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 02:49:01.447684 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441832 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 02:49:01.447684 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441834 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 02:49:01.448157 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441837 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 02:49:01.448157 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441839 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 02:49:01.448157 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441843 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 02:49:01.448157 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441847 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 02:49:01.448157 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441850 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 02:49:01.448157 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441852 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 02:49:01.448157 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441855 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 02:49:01.448157 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441858 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 02:49:01.448157 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441861 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 02:49:01.448157 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441864 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 02:49:01.448157 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441866 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 02:49:01.448157 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441869 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 02:49:01.448157 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441871 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 02:49:01.448157 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441874 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 02:49:01.448157 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441877 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 02:49:01.448157 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441880 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 02:49:01.448157 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441882 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 02:49:01.448157 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:01.441885 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 02:49:01.448617 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.441890 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 02:49:01.448617 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.442021 2560 server.go:962] "Client rotation is on, will bootstrap in background" Sep 10 02:49:01.448617 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.444210 2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 10 02:49:01.448617 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.445197 2560 server.go:1019] "Starting client certificate rotation" Sep 10 02:49:01.448617 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.445300 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 02:49:01.448617 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.445987 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 02:49:01.448617 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.448320 2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 10 02:49:01.453456 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.453432 2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 10 02:49:01.462737 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.462715 2560 log.go:25] "Validated CRI v1 runtime API" Sep 10 02:49:01.465609 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.465592 2560 log.go:25] "Validated CRI v1 image API" Sep 10 02:49:01.466418 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.466401 2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 10 02:49:01.468598 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.468579 2560 fs.go:135] Filesystem UUIDs: map[264bcdb9-7bf4-4236-92bd-0a3f27ccdfe7:/dev/nvme0n1p4 61ce41ed-1f0f-40ee-b8fe-7b28e37fff56:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Sep 10 02:49:01.468646 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.468598 2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 10 02:49:01.474017 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.473906 2560 manager.go:216] Machine: {Timestamp:2026-09-10 02:49:01.472522892 +0000 UTC m=+0.319352151 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3161403 MemoryCapacity:32812163072 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2201d29b595fb7a89b4f6fc4db287b SystemUUID:ec2201d2-9b59-5fb7-a89b-4f6fc4db287b BootID:d43dd9c6-a539-43a2-8857-2e97d5cdc424 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:a9:4a:a0:d4:ed Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:a9:4a:a0:d4:ed Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a2:1b:66:0c:94:07 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812163072 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 10 02:49:01.474017 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.474016 2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 10 02:49:01.474162 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.474106 2560 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 10 02:49:01.475536 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.475501 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 10 02:49:01.477023 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.476997 2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 10 02:49:01.477178 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.477027 2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-37.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 10 02:49:01.477801 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.477789 2560 topology_manager.go:138] "Creating topology manager with none policy" Sep 10 02:49:01.477843 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.477805 2560 container_manager_linux.go:309] "Creating device plugin manager" Sep 10 02:49:01.477843 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.477813 2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 10 02:49:01.478532 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.478522 2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 10 02:49:01.479838 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.479829 2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 10 02:49:01.484453 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.484440 2560 state_mem.go:36] "Initialized new in-memory state store" Sep 10 02:49:01.484600 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.484590 2560 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 10 02:49:01.484633 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.484627 2560 kubelet.go:486] "Attempting to sync node with API server" Sep 10 02:49:01.484668 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.484641 2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 10 02:49:01.484668 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.484655 2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 10 02:49:01.484668 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.484665 2560 kubelet.go:398] "Adding apiserver pod source" Sep 10 02:49:01.484744 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.484674 2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 10 02:49:01.485134 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.485120 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 10 02:49:01.485176 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.485145 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 10 02:49:01.487404 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.487386 2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 10 02:49:01.489863 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.489847 2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 10 02:49:01.489942 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.489886 2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 10 02:49:01.490112 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.490097 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 10 02:49:01.490169 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.490119 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 10 02:49:01.490169 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.490128 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 10 02:49:01.490169 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.490137 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 10 02:49:01.490169 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.490149 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 10 02:49:01.490169 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.490159 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 10 02:49:01.490634 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.490621 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 10 02:49:01.490699 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.490637 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 10 02:49:01.490699 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.490648 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 10 02:49:01.490699 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.490658 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 10 02:49:01.490699 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.490689 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 10 02:49:01.490837 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.490708 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 10 02:49:01.491447 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.491437 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 10 02:49:01.491496 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.491450 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 10 02:49:01.495847 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.495473 2560 server.go:1268] "Started kubelet" Sep 10 02:49:01.495947 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.495727 2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 10 02:49:01.496091 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.496077 2560 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 10 02:49:01.496615 ip-10-0-133-37 systemd[1]: Started Kubernetes Kubelet. Sep 10 02:49:01.497051 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.497022 2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 10 02:49:01.497923 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.497440 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-37.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 02:49:01.497923 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:01.497664 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-37.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 10 02:49:01.497923 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:01.497707 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 10 02:49:01.497923 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.497211 2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 10 02:49:01.499812 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.499795 2560 server.go:310] "Adding debug handlers to kubelet server" Sep 10 02:49:01.501862 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:01.501842 2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 10 02:49:01.502141 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.502119 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 10 02:49:01.502905 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.502887 2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 10 02:49:01.503041 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:01.502992 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-37.ec2.internal\" not found" Sep 10 02:49:01.503557 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.503539 2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 10 02:49:01.503557 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.503554 2560 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 10 02:49:01.503678 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.503569 2560 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 10 02:49:01.503678 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.503637 2560 reconstruct.go:97] "Volume reconstruction finished" Sep 10 02:49:01.503678 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.503647 2560 reconciler.go:29] "Reconciler: start to sync state" Sep 10 02:49:01.503771 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.503725 2560 factory.go:55] Registering systemd factory Sep 10 02:49:01.503771 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.503736 2560 factory.go:223] Registration of the systemd container factory successfully Sep 10 02:49:01.503940 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.503928 2560 factory.go:153] Registering CRI-O factory Sep 10 02:49:01.504002 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.503945 2560 factory.go:223] Registration of the crio container factory successfully Sep 10 02:49:01.504002 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.503998 2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 10 02:49:01.504087 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.504018 2560 factory.go:103] Registering Raw factory Sep 10 02:49:01.504087 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.504034 2560 manager.go:1196] Started watching for new ooms in manager Sep 10 02:49:01.504372 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.504359 2560 manager.go:349] Starting recovery of all containers Sep 10 02:49:01.505026 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:01.505005 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 10 02:49:01.506072 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:01.505043 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-37.ec2.internal.18d3d5ca37f95398 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-37.ec2.internal,UID:ip-10-0-133-37.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-37.ec2.internal,},FirstTimestamp:2026-09-10 02:49:01.495415704 +0000 UTC m=+0.342244963,LastTimestamp:2026-09-10 02:49:01.495415704 +0000 UTC m=+0.342244963,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-37.ec2.internal,}" Sep 10 02:49:01.506072 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:01.506001 2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-37.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 10 02:49:01.506072 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.515130 2560 manager.go:354] Recovery completed Sep 10 02:49:01.520877 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.520861 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 02:49:01.523262 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.523233 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-37.ec2.internal" event="NodeHasSufficientMemory" Sep 10 02:49:01.523349 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.523275 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-37.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 02:49:01.523349 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.523291 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-37.ec2.internal" event="NodeHasSufficientPID" Sep 10 02:49:01.523900 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.523884 2560 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 10 02:49:01.523900 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.523897 2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 10 02:49:01.524011 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.523930 2560 state_mem.go:36] "Initialized new in-memory state store" Sep 10 02:49:01.524011 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.523930 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 02:49:01.524735 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.524718 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-37.ec2.internal" event="NodeHasSufficientMemory" Sep 10 02:49:01.524811 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.524745 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-37.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 02:49:01.524811 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.524755 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-37.ec2.internal" event="NodeHasSufficientPID" Sep 10 02:49:01.526211 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.526196 2560 policy_none.go:49] "None policy: Start" Sep 10 02:49:01.526264 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.526217 2560 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 10 02:49:01.526264 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.526231 2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 10 02:49:01.528262 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.528247 2560 policy_none.go:47] "Start" Sep 10 02:49:01.535797 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:01.535174 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-37.ec2.internal.18d3d5ca39a233ac default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-37.ec2.internal,UID:ip-10-0-133-37.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-37.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-37.ec2.internal,},FirstTimestamp:2026-09-10 02:49:01.523260332 +0000 UTC m=+0.370089591,LastTimestamp:2026-09-10 02:49:01.523260332 +0000 UTC m=+0.370089591,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-37.ec2.internal,}" Sep 10 02:49:01.537305 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.537285 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-h5clv" Sep 10 02:49:01.547446 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.547418 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-h5clv" Sep 10 02:49:01.549322 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:01.549150 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-37.ec2.internal.18d3d5ca39a28ae9 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-37.ec2.internal,UID:ip-10-0-133-37.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-37.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-37.ec2.internal,},FirstTimestamp:2026-09-10 02:49:01.523282665 +0000 UTC m=+0.370111929,LastTimestamp:2026-09-10 02:49:01.523282665 +0000 UTC m=+0.370111929,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-37.ec2.internal,}" Sep 10 02:49:01.555213 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.555188 2560 manager.go:337] "Starting Device Plugin manager" Sep 10 02:49:01.555294 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:01.555237 2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 10 02:49:01.555294 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.555251 2560 server.go:87] "Starting device plugin registration server" Sep 10 02:49:01.555542 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.555529 2560 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 10 02:49:01.555601 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.555552 2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 10 02:49:01.555786 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.555725 2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 10 02:49:01.555848 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.555822 2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 10 02:49:01.555848 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.555832 2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 10 02:49:01.556323 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:01.556302 2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 10 02:49:01.556408 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:01.556376 2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-37.ec2.internal\" not found" Sep 10 02:49:01.637895 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.637855 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 10 02:49:01.640030 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.639400 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 10 02:49:01.640030 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.639448 2560 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 10 02:49:01.640030 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.639477 2560 kubelet.go:2448] "Starting kubelet main sync loop" Sep 10 02:49:01.640030 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:01.639590 2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 10 02:49:01.642291 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.642273 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 02:49:01.656380 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.656356 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 02:49:01.657338 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.657324 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-37.ec2.internal" event="NodeHasSufficientMemory" Sep 10 02:49:01.657410 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.657356 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-37.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 02:49:01.657410 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.657367 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-37.ec2.internal" event="NodeHasSufficientPID" Sep 10 02:49:01.657410 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.657394 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-37.ec2.internal" Sep 10 02:49:01.667625 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.667605 2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-37.ec2.internal" Sep 10 02:49:01.667690 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:01.667634 2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-37.ec2.internal\": node \"ip-10-0-133-37.ec2.internal\" not found" Sep 10 02:49:01.697975 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:01.697904 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-37.ec2.internal\" not found" Sep 10 02:49:01.740034 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.739999 2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-37.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-37.ec2.internal"] Sep 10 02:49:01.740190 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.740098 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 02:49:01.741659 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.741641 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-37.ec2.internal" event="NodeHasSufficientMemory" Sep 10 02:49:01.741762 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.741677 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-37.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 02:49:01.741762 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.741690 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-37.ec2.internal" event="NodeHasSufficientPID" Sep 10 02:49:01.743978 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.743962 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 02:49:01.744086 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.744069 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-37.ec2.internal" Sep 10 02:49:01.744138 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.744103 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 02:49:01.744758 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.744736 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-37.ec2.internal" event="NodeHasSufficientMemory" Sep 10 02:49:01.744847 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.744762 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-37.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 02:49:01.744847 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.744776 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-37.ec2.internal" event="NodeHasSufficientPID" Sep 10 02:49:01.744847 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.744784 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-37.ec2.internal" event="NodeHasSufficientMemory" Sep 10 02:49:01.744847 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.744805 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-37.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 02:49:01.744847 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.744816 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-37.ec2.internal" event="NodeHasSufficientPID" Sep 10 02:49:01.746997 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.746980 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-37.ec2.internal" Sep 10 02:49:01.747079 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.747012 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 02:49:01.747835 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.747798 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-37.ec2.internal" event="NodeHasSufficientMemory" Sep 10 02:49:01.747835 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.747833 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-37.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 02:49:01.747949 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.747847 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-37.ec2.internal" event="NodeHasSufficientPID" Sep 10 02:49:01.769195 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:01.769172 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-37.ec2.internal\" not found" node="ip-10-0-133-37.ec2.internal" Sep 10 02:49:01.773858 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:01.773841 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-37.ec2.internal\" not found" node="ip-10-0-133-37.ec2.internal" Sep 10 02:49:01.798780 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:01.798749 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-37.ec2.internal\" not found" Sep 10 02:49:01.805276 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.805253 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8da85c75381fab0440e1fd992aacf5e7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-37.ec2.internal\" (UID: \"8da85c75381fab0440e1fd992aacf5e7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-37.ec2.internal" Sep 10 02:49:01.805379 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.805283 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8da85c75381fab0440e1fd992aacf5e7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-37.ec2.internal\" (UID: \"8da85c75381fab0440e1fd992aacf5e7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-37.ec2.internal" Sep 10 02:49:01.805379 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.805298 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/480faa79f347b524a5131c7d80428e3a-config\") pod \"kube-apiserver-proxy-ip-10-0-133-37.ec2.internal\" (UID: \"480faa79f347b524a5131c7d80428e3a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-37.ec2.internal" Sep 10 02:49:01.899795 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:01.899743 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-37.ec2.internal\" not found" Sep 10 02:49:01.906107 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.906072 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8da85c75381fab0440e1fd992aacf5e7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-37.ec2.internal\" (UID: \"8da85c75381fab0440e1fd992aacf5e7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-37.ec2.internal" Sep 10 02:49:01.906107 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.906107 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8da85c75381fab0440e1fd992aacf5e7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-37.ec2.internal\" (UID: \"8da85c75381fab0440e1fd992aacf5e7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-37.ec2.internal" Sep 10 02:49:01.906272 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.906132 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/480faa79f347b524a5131c7d80428e3a-config\") pod \"kube-apiserver-proxy-ip-10-0-133-37.ec2.internal\" (UID: \"480faa79f347b524a5131c7d80428e3a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-37.ec2.internal" Sep 10 02:49:01.906272 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.906198 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8da85c75381fab0440e1fd992aacf5e7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-37.ec2.internal\" (UID: \"8da85c75381fab0440e1fd992aacf5e7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-37.ec2.internal" Sep 10 02:49:01.906272 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.906215 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8da85c75381fab0440e1fd992aacf5e7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-37.ec2.internal\" (UID: \"8da85c75381fab0440e1fd992aacf5e7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-37.ec2.internal" Sep 10 02:49:01.906272 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:01.906200 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/480faa79f347b524a5131c7d80428e3a-config\") pod \"kube-apiserver-proxy-ip-10-0-133-37.ec2.internal\" (UID: \"480faa79f347b524a5131c7d80428e3a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-37.ec2.internal" Sep 10 02:49:02.000629 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:02.000524 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-37.ec2.internal\" not found" Sep 10 02:49:02.072048 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:02.072018 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-37.ec2.internal" Sep 10 02:49:02.077300 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:02.076964 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-37.ec2.internal" Sep 10 02:49:02.101544 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:02.101495 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-37.ec2.internal\" not found" Sep 10 02:49:02.202064 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:02.202017 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-37.ec2.internal\" not found" Sep 10 02:49:02.302615 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:02.302533 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-37.ec2.internal\" not found" Sep 10 02:49:02.403014 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:02.402961 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-37.ec2.internal\" not found" Sep 10 02:49:02.445388 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:02.445353 2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 10 02:49:02.445990 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:02.445534 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 10 02:49:02.504061 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:02.504020 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-37.ec2.internal\" not found" Sep 10 02:49:02.543153 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:02.543124 2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 02:49:02.551119 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:02.551076 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 02:44:02 +0000 UTC" deadline="2028-04-02 03:15:30.055723281 +0000 UTC" Sep 10 02:49:02.551119 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:02.551115 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13680h26m27.504611127s" Sep 10 02:49:02.604655 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:02.604560 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-37.ec2.internal\" not found" Sep 10 02:49:02.631471 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:02.631440 2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 02:49:02.685832 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:02.685171 2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 02:49:02.704166 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:02.704142 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-37.ec2.internal" Sep 10 02:49:02.719005 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:02.718983 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 10 02:49:02.720011 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:02.719998 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-37.ec2.internal" Sep 10 02:49:02.727445 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:02.727429 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 10 02:49:02.764941 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:02.764910 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod480faa79f347b524a5131c7d80428e3a.slice/crio-de6a730bb61fef1a0d0bffcf1e564d8131d1317501b2f28cd9ae5d67e0437d8c WatchSource:0}: Error finding container de6a730bb61fef1a0d0bffcf1e564d8131d1317501b2f28cd9ae5d67e0437d8c: Status 404 returned error can't find the container with id de6a730bb61fef1a0d0bffcf1e564d8131d1317501b2f28cd9ae5d67e0437d8c Sep 10 02:49:02.765219 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:02.765199 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8da85c75381fab0440e1fd992aacf5e7.slice/crio-4def1138106d677d7bacdbd1da5e3150d6f4384c8fa06488dfcfb1770fc3ac9a WatchSource:0}: Error finding container 4def1138106d677d7bacdbd1da5e3150d6f4384c8fa06488dfcfb1770fc3ac9a: Status 404 returned error can't find the container with id 4def1138106d677d7bacdbd1da5e3150d6f4384c8fa06488dfcfb1770fc3ac9a Sep 10 02:49:02.769911 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:02.769885 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 02:49:03.463151 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.463055 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 02:49:03.486642 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.486604 2560 apiserver.go:52] "Watching apiserver" Sep 10 02:49:03.494608 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.494576 2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 10 02:49:03.496794 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.496760 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-kkzdh","openshift-image-registry/node-ca-mnpps","openshift-multus/multus-additional-cni-plugins-6d6zf","openshift-multus/network-metrics-daemon-d2sbz","openshift-ovn-kubernetes/ovnkube-node-swhbm","kube-system/konnectivity-agent-z9d9b","kube-system/kube-apiserver-proxy-ip-10-0-133-37.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p6kp7","openshift-cluster-node-tuning-operator/tuned-xbp79","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-37.ec2.internal","openshift-multus/multus-28crk","openshift-network-diagnostics/network-check-target-fqfck","openshift-network-operator/iptables-alerter-fwtq4","kube-system/global-pull-secret-syncer-sdxng"] Sep 10 02:49:03.500371 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.499687 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fwtq4" Sep 10 02:49:03.502187 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.502018 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mnpps" Sep 10 02:49:03.503179 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.503139 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 10 02:49:03.504707 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.504543 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6d6zf" Sep 10 02:49:03.507277 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.506966 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2sbz" Sep 10 02:49:03.507277 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:03.507029 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d2sbz" podUID="c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e" Sep 10 02:49:03.510861 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.509781 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.514106 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.511728 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 10 02:49:03.514106 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.512024 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 10 02:49:03.514106 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.512214 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 10 02:49:03.514106 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.512515 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 10 02:49:03.514106 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.513017 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 10 02:49:03.514106 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.513235 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 10 02:49:03.514106 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.513385 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fj8jr\" (UniqueName: \"kubernetes.io/projected/d2635452-ab19-4377-93d4-3db8d6e0aa88-kube-api-access-fj8jr\") pod \"multus-additional-cni-plugins-6d6zf\" (UID: \"d2635452-ab19-4377-93d4-3db8d6e0aa88\") " pod="openshift-multus/multus-additional-cni-plugins-6d6zf" Sep 10 02:49:03.514106 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.513415 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d2635452-ab19-4377-93d4-3db8d6e0aa88-cni-binary-copy\") pod \"multus-additional-cni-plugins-6d6zf\" (UID: \"d2635452-ab19-4377-93d4-3db8d6e0aa88\") " pod="openshift-multus/multus-additional-cni-plugins-6d6zf" Sep 10 02:49:03.514106 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.513482 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mr5s\" (UniqueName: \"kubernetes.io/projected/078d27a8-8e62-4092-943b-ade84d558613-kube-api-access-5mr5s\") pod \"iptables-alerter-fwtq4\" (UID: \"078d27a8-8e62-4092-943b-ade84d558613\") " pod="openshift-network-operator/iptables-alerter-fwtq4" Sep 10 02:49:03.514106 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.513509 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbrm7\" (UniqueName: \"kubernetes.io/projected/936410ac-13a7-43ca-a51f-b0bd96649ff6-kube-api-access-tbrm7\") pod \"node-ca-mnpps\" (UID: \"936410ac-13a7-43ca-a51f-b0bd96649ff6\") " pod="openshift-image-registry/node-ca-mnpps" Sep 10 02:49:03.514106 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.513543 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d2635452-ab19-4377-93d4-3db8d6e0aa88-cnibin\") pod \"multus-additional-cni-plugins-6d6zf\" (UID: \"d2635452-ab19-4377-93d4-3db8d6e0aa88\") " pod="openshift-multus/multus-additional-cni-plugins-6d6zf" Sep 10 02:49:03.514106 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.513572 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d2635452-ab19-4377-93d4-3db8d6e0aa88-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6d6zf\" (UID: \"d2635452-ab19-4377-93d4-3db8d6e0aa88\") " pod="openshift-multus/multus-additional-cni-plugins-6d6zf" Sep 10 02:49:03.514106 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.513616 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d2635452-ab19-4377-93d4-3db8d6e0aa88-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6d6zf\" (UID: \"d2635452-ab19-4377-93d4-3db8d6e0aa88\") " pod="openshift-multus/multus-additional-cni-plugins-6d6zf" Sep 10 02:49:03.514106 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.513645 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/078d27a8-8e62-4092-943b-ade84d558613-iptables-alerter-script\") pod \"iptables-alerter-fwtq4\" (UID: \"078d27a8-8e62-4092-943b-ade84d558613\") " pod="openshift-network-operator/iptables-alerter-fwtq4" Sep 10 02:49:03.514106 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.513686 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/078d27a8-8e62-4092-943b-ade84d558613-host-slash\") pod \"iptables-alerter-fwtq4\" (UID: \"078d27a8-8e62-4092-943b-ade84d558613\") " pod="openshift-network-operator/iptables-alerter-fwtq4" Sep 10 02:49:03.514106 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.513710 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/936410ac-13a7-43ca-a51f-b0bd96649ff6-host\") pod \"node-ca-mnpps\" (UID: \"936410ac-13a7-43ca-a51f-b0bd96649ff6\") " pod="openshift-image-registry/node-ca-mnpps" Sep 10 02:49:03.514106 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.513754 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/936410ac-13a7-43ca-a51f-b0bd96649ff6-serviceca\") pod \"node-ca-mnpps\" (UID: \"936410ac-13a7-43ca-a51f-b0bd96649ff6\") " pod="openshift-image-registry/node-ca-mnpps" Sep 10 02:49:03.514106 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.513791 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d2635452-ab19-4377-93d4-3db8d6e0aa88-system-cni-dir\") pod \"multus-additional-cni-plugins-6d6zf\" (UID: \"d2635452-ab19-4377-93d4-3db8d6e0aa88\") " pod="openshift-multus/multus-additional-cni-plugins-6d6zf" Sep 10 02:49:03.514106 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.513815 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d2635452-ab19-4377-93d4-3db8d6e0aa88-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6d6zf\" (UID: \"d2635452-ab19-4377-93d4-3db8d6e0aa88\") " pod="openshift-multus/multus-additional-cni-plugins-6d6zf" Sep 10 02:49:03.514106 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.513822 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-z9d9b" Sep 10 02:49:03.516261 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.515842 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kkzdh" Sep 10 02:49:03.519420 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.518682 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p6kp7" Sep 10 02:49:03.519420 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.518899 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4hhk9\"" Sep 10 02:49:03.519838 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.519625 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 10 02:49:03.527608 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.522098 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.532128 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.532103 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sdxng" Sep 10 02:49:03.532240 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:03.532190 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sdxng" podUID="9ff1e29e-7033-490c-9316-def0076b8f7e" Sep 10 02:49:03.535092 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.535071 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-28crk" Sep 10 02:49:03.537392 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.537273 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 10 02:49:03.537572 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.537549 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 10 02:49:03.537572 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.537611 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fqfck" Sep 10 02:49:03.537572 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:03.537666 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fqfck" podUID="690238d6-9e65-42b8-b042-d1c3efbc1e45" Sep 10 02:49:03.538138 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.538024 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 10 02:49:03.538138 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.538031 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 10 02:49:03.538138 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.538044 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 10 02:49:03.538359 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.538331 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 10 02:49:03.538429 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.538380 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-wn9v4\"" Sep 10 02:49:03.538715 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.538699 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 10 02:49:03.538813 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.538734 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 10 02:49:03.539214 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.539197 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 10 02:49:03.539440 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.539233 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 10 02:49:03.539746 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.539730 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 10 02:49:03.539972 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.539956 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 10 02:49:03.540312 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.540294 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 10 02:49:03.540650 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.540636 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 10 02:49:03.540868 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.540849 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-kk2pb\"" Sep 10 02:49:03.541151 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.541127 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 10 02:49:03.541279 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.541127 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 10 02:49:03.541456 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.541435 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hk2hr\"" Sep 10 02:49:03.541575 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.541335 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-8gfj6\"" Sep 10 02:49:03.541874 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.541857 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 10 02:49:03.542354 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.542337 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 10 02:49:03.551580 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.551086 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-pj658\"" Sep 10 02:49:03.551580 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.551318 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 10 02:49:03.551580 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.551334 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-plbfk\"" Sep 10 02:49:03.551580 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.551345 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 10 02:49:03.551580 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.551460 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-gch8l\"" Sep 10 02:49:03.551894 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.551628 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7f9h2\"" Sep 10 02:49:03.560028 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.559998 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 10 02:49:03.600747 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.600720 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-kpzbd" Sep 10 02:49:03.604999 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.604977 2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 10 02:49:03.614019 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.613991 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-host-slash\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.614149 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.614031 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d2635452-ab19-4377-93d4-3db8d6e0aa88-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6d6zf\" (UID: \"d2635452-ab19-4377-93d4-3db8d6e0aa88\") " pod="openshift-multus/multus-additional-cni-plugins-6d6zf" Sep 10 02:49:03.614149 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.614060 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c0f2ef31-35db-4d61-b0c4-77358e82154f-etc-sysctl-d\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.614149 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.614093 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c0f2ef31-35db-4d61-b0c4-77358e82154f-tmp\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.614149 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.614113 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-multus-conf-dir\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.614149 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.614135 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-run-openvswitch\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.614371 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.614155 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/76f77578-08ad-40d2-849d-7d71c6b7d76b-device-dir\") pod \"aws-ebs-csi-driver-node-p6kp7\" (UID: \"76f77578-08ad-40d2-849d-7d71c6b7d76b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p6kp7" Sep 10 02:49:03.614371 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.614177 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d2635452-ab19-4377-93d4-3db8d6e0aa88-cni-binary-copy\") pod \"multus-additional-cni-plugins-6d6zf\" (UID: \"d2635452-ab19-4377-93d4-3db8d6e0aa88\") " pod="openshift-multus/multus-additional-cni-plugins-6d6zf" Sep 10 02:49:03.614371 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.614212 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9ff1e29e-7033-490c-9316-def0076b8f7e-dbus\") pod \"global-pull-secret-syncer-sdxng\" (UID: \"9ff1e29e-7033-490c-9316-def0076b8f7e\") " pod="kube-system/global-pull-secret-syncer-sdxng" Sep 10 02:49:03.614371 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.614238 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-system-cni-dir\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.614371 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.614262 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-etc-kubernetes\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.614371 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.614287 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-host-kubelet\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.614371 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.614330 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/76f77578-08ad-40d2-849d-7d71c6b7d76b-registration-dir\") pod \"aws-ebs-csi-driver-node-p6kp7\" (UID: \"76f77578-08ad-40d2-849d-7d71c6b7d76b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p6kp7" Sep 10 02:49:03.614758 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.614356 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7r8h\" (UniqueName: \"kubernetes.io/projected/76f77578-08ad-40d2-849d-7d71c6b7d76b-kube-api-access-v7r8h\") pod \"aws-ebs-csi-driver-node-p6kp7\" (UID: \"76f77578-08ad-40d2-849d-7d71c6b7d76b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p6kp7" Sep 10 02:49:03.614758 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.614405 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-hostroot\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.614758 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.614431 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-run-systemd\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.614758 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.614456 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/76f77578-08ad-40d2-849d-7d71c6b7d76b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p6kp7\" (UID: \"76f77578-08ad-40d2-849d-7d71c6b7d76b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p6kp7" Sep 10 02:49:03.614758 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.614484 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/76f77578-08ad-40d2-849d-7d71c6b7d76b-sys-fs\") pod \"aws-ebs-csi-driver-node-p6kp7\" (UID: \"76f77578-08ad-40d2-849d-7d71c6b7d76b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p6kp7" Sep 10 02:49:03.614758 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.614511 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c0f2ef31-35db-4d61-b0c4-77358e82154f-lib-modules\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.614758 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.614543 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/078d27a8-8e62-4092-943b-ade84d558613-iptables-alerter-script\") pod \"iptables-alerter-fwtq4\" (UID: \"078d27a8-8e62-4092-943b-ade84d558613\") " pod="openshift-network-operator/iptables-alerter-fwtq4" Sep 10 02:49:03.614758 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.614589 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-multus-socket-dir-parent\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.614758 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.614617 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-host-var-lib-cni-bin\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.614758 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.614646 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-etc-openvswitch\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.614758 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.614671 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6fdc2579-93fb-494d-b179-46cf1db9aaa8-hosts-file\") pod \"node-resolver-kkzdh\" (UID: \"6fdc2579-93fb-494d-b179-46cf1db9aaa8\") " pod="openshift-dns/node-resolver-kkzdh" Sep 10 02:49:03.614758 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.614703 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5bhr\" (UniqueName: \"kubernetes.io/projected/6fdc2579-93fb-494d-b179-46cf1db9aaa8-kube-api-access-x5bhr\") pod \"node-resolver-kkzdh\" (UID: \"6fdc2579-93fb-494d-b179-46cf1db9aaa8\") " pod="openshift-dns/node-resolver-kkzdh" Sep 10 02:49:03.614758 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.614731 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-host-run-multus-certs\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.614758 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.614764 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-host-cni-bin\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.615436 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.614789 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8a6680c4-21b8-4fe2-a763-e84858c7ef90-env-overrides\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.615436 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.614815 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8a6680c4-21b8-4fe2-a763-e84858c7ef90-ovn-node-metrics-cert\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.615436 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.614848 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/27be8319-a821-4e27-9dc8-411257676cb7-agent-certs\") pod \"konnectivity-agent-z9d9b\" (UID: \"27be8319-a821-4e27-9dc8-411257676cb7\") " pod="kube-system/konnectivity-agent-z9d9b" Sep 10 02:49:03.615436 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.614881 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d2635452-ab19-4377-93d4-3db8d6e0aa88-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6d6zf\" (UID: \"d2635452-ab19-4377-93d4-3db8d6e0aa88\") " pod="openshift-multus/multus-additional-cni-plugins-6d6zf" Sep 10 02:49:03.615436 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.614908 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c0f2ef31-35db-4d61-b0c4-77358e82154f-etc-tuned\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.615436 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.614931 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/27be8319-a821-4e27-9dc8-411257676cb7-konnectivity-ca\") pod \"konnectivity-agent-z9d9b\" (UID: \"27be8319-a821-4e27-9dc8-411257676cb7\") " pod="kube-system/konnectivity-agent-z9d9b" Sep 10 02:49:03.615436 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.614957 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/76f77578-08ad-40d2-849d-7d71c6b7d76b-etc-selinux\") pod \"aws-ebs-csi-driver-node-p6kp7\" (UID: \"76f77578-08ad-40d2-849d-7d71c6b7d76b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p6kp7" Sep 10 02:49:03.615436 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.614986 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smjcj\" (UniqueName: \"kubernetes.io/projected/c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e-kube-api-access-smjcj\") pod \"network-metrics-daemon-d2sbz\" (UID: \"c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e\") " pod="openshift-multus/network-metrics-daemon-d2sbz" Sep 10 02:49:03.615436 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615011 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-host-cni-netd\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.615436 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615038 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d2635452-ab19-4377-93d4-3db8d6e0aa88-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6d6zf\" (UID: \"d2635452-ab19-4377-93d4-3db8d6e0aa88\") " pod="openshift-multus/multus-additional-cni-plugins-6d6zf" Sep 10 02:49:03.615436 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615064 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/078d27a8-8e62-4092-943b-ade84d558613-host-slash\") pod \"iptables-alerter-fwtq4\" (UID: \"078d27a8-8e62-4092-943b-ade84d558613\") " pod="openshift-network-operator/iptables-alerter-fwtq4" Sep 10 02:49:03.615436 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615100 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c0f2ef31-35db-4d61-b0c4-77358e82154f-etc-sysctl-conf\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.615436 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615126 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drnkg\" (UniqueName: \"kubernetes.io/projected/c0f2ef31-35db-4d61-b0c4-77358e82154f-kube-api-access-drnkg\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.615436 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615151 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8c69b77f-74ff-4667-9c26-9beba2e52e21-multus-daemon-config\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.615436 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615187 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-host-run-netns\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.615436 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615214 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/936410ac-13a7-43ca-a51f-b0bd96649ff6-host\") pod \"node-ca-mnpps\" (UID: \"936410ac-13a7-43ca-a51f-b0bd96649ff6\") " pod="openshift-image-registry/node-ca-mnpps" Sep 10 02:49:03.615963 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615243 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d2635452-ab19-4377-93d4-3db8d6e0aa88-system-cni-dir\") pod \"multus-additional-cni-plugins-6d6zf\" (UID: \"d2635452-ab19-4377-93d4-3db8d6e0aa88\") " pod="openshift-multus/multus-additional-cni-plugins-6d6zf" Sep 10 02:49:03.615963 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615270 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fj8jr\" (UniqueName: \"kubernetes.io/projected/d2635452-ab19-4377-93d4-3db8d6e0aa88-kube-api-access-fj8jr\") pod \"multus-additional-cni-plugins-6d6zf\" (UID: \"d2635452-ab19-4377-93d4-3db8d6e0aa88\") " pod="openshift-multus/multus-additional-cni-plugins-6d6zf" Sep 10 02:49:03.615963 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615296 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ff1e29e-7033-490c-9316-def0076b8f7e-original-pull-secret\") pod \"global-pull-secret-syncer-sdxng\" (UID: \"9ff1e29e-7033-490c-9316-def0076b8f7e\") " pod="kube-system/global-pull-secret-syncer-sdxng" Sep 10 02:49:03.615963 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615315 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-run-ovn\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.615963 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615332 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8a6680c4-21b8-4fe2-a763-e84858c7ef90-ovnkube-script-lib\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.615963 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615348 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tz692\" (UniqueName: \"kubernetes.io/projected/8a6680c4-21b8-4fe2-a763-e84858c7ef90-kube-api-access-tz692\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.615963 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615368 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mr5s\" (UniqueName: \"kubernetes.io/projected/078d27a8-8e62-4092-943b-ade84d558613-kube-api-access-5mr5s\") pod \"iptables-alerter-fwtq4\" (UID: \"078d27a8-8e62-4092-943b-ade84d558613\") " pod="openshift-network-operator/iptables-alerter-fwtq4" Sep 10 02:49:03.615963 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615384 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d2635452-ab19-4377-93d4-3db8d6e0aa88-cnibin\") pod \"multus-additional-cni-plugins-6d6zf\" (UID: \"d2635452-ab19-4377-93d4-3db8d6e0aa88\") " pod="openshift-multus/multus-additional-cni-plugins-6d6zf" Sep 10 02:49:03.615963 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615401 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c0f2ef31-35db-4d61-b0c4-77358e82154f-etc-kubernetes\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.615963 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615426 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8shw\" (UniqueName: \"kubernetes.io/projected/8c69b77f-74ff-4667-9c26-9beba2e52e21-kube-api-access-l8shw\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.615963 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615448 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8a6680c4-21b8-4fe2-a763-e84858c7ef90-ovnkube-config\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.615963 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615464 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fdc2579-93fb-494d-b179-46cf1db9aaa8-tmp-dir\") pod \"node-resolver-kkzdh\" (UID: \"6fdc2579-93fb-494d-b179-46cf1db9aaa8\") " pod="openshift-dns/node-resolver-kkzdh" Sep 10 02:49:03.615963 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615479 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c0f2ef31-35db-4d61-b0c4-77358e82154f-sys\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.615963 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615494 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c0f2ef31-35db-4d61-b0c4-77358e82154f-host\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.615963 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615517 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-cnibin\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.615963 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615536 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-host-var-lib-cni-multus\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.615963 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615556 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-host-var-lib-kubelet\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.616915 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615572 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c0f2ef31-35db-4d61-b0c4-77358e82154f-etc-systemd\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.616915 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615608 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c0f2ef31-35db-4d61-b0c4-77358e82154f-etc-modprobe-d\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.616915 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615631 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-host-run-k8s-cni-cncf-io\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.616915 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615654 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/936410ac-13a7-43ca-a51f-b0bd96649ff6-serviceca\") pod \"node-ca-mnpps\" (UID: \"936410ac-13a7-43ca-a51f-b0bd96649ff6\") " pod="openshift-image-registry/node-ca-mnpps" Sep 10 02:49:03.616915 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615679 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-host-run-netns\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.616915 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615705 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.616915 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615728 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbrm7\" (UniqueName: \"kubernetes.io/projected/936410ac-13a7-43ca-a51f-b0bd96649ff6-kube-api-access-tbrm7\") pod \"node-ca-mnpps\" (UID: \"936410ac-13a7-43ca-a51f-b0bd96649ff6\") " pod="openshift-image-registry/node-ca-mnpps" Sep 10 02:49:03.616915 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615749 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e-metrics-certs\") pod \"network-metrics-daemon-d2sbz\" (UID: \"c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e\") " pod="openshift-multus/network-metrics-daemon-d2sbz" Sep 10 02:49:03.616915 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615776 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c0f2ef31-35db-4d61-b0c4-77358e82154f-run\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.616915 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615803 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9ff1e29e-7033-490c-9316-def0076b8f7e-kubelet-config\") pod \"global-pull-secret-syncer-sdxng\" (UID: \"9ff1e29e-7033-490c-9316-def0076b8f7e\") " pod="kube-system/global-pull-secret-syncer-sdxng" Sep 10 02:49:03.616915 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615830 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-multus-cni-dir\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.616915 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615858 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-host-run-ovn-kubernetes\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.616915 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615886 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c0f2ef31-35db-4d61-b0c4-77358e82154f-var-lib-kubelet\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.616915 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615915 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8c69b77f-74ff-4667-9c26-9beba2e52e21-cni-binary-copy\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.616915 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615944 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-systemd-units\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.616915 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615972 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-var-lib-openvswitch\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.616915 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.615999 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-log-socket\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.617764 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.616033 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c0f2ef31-35db-4d61-b0c4-77358e82154f-etc-sysconfig\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.617764 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.616059 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pt95b\" (UniqueName: \"kubernetes.io/projected/690238d6-9e65-42b8-b042-d1c3efbc1e45-kube-api-access-pt95b\") pod \"network-check-target-fqfck\" (UID: \"690238d6-9e65-42b8-b042-d1c3efbc1e45\") " pod="openshift-network-diagnostics/network-check-target-fqfck" Sep 10 02:49:03.617764 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.616076 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-node-log\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.617764 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.616094 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/76f77578-08ad-40d2-849d-7d71c6b7d76b-socket-dir\") pod \"aws-ebs-csi-driver-node-p6kp7\" (UID: \"76f77578-08ad-40d2-849d-7d71c6b7d76b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p6kp7" Sep 10 02:49:03.617764 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.616494 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d2635452-ab19-4377-93d4-3db8d6e0aa88-cnibin\") pod \"multus-additional-cni-plugins-6d6zf\" (UID: \"d2635452-ab19-4377-93d4-3db8d6e0aa88\") " pod="openshift-multus/multus-additional-cni-plugins-6d6zf" Sep 10 02:49:03.617764 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.616567 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d2635452-ab19-4377-93d4-3db8d6e0aa88-system-cni-dir\") pod \"multus-additional-cni-plugins-6d6zf\" (UID: \"d2635452-ab19-4377-93d4-3db8d6e0aa88\") " pod="openshift-multus/multus-additional-cni-plugins-6d6zf" Sep 10 02:49:03.617764 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.616654 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/078d27a8-8e62-4092-943b-ade84d558613-host-slash\") pod \"iptables-alerter-fwtq4\" (UID: \"078d27a8-8e62-4092-943b-ade84d558613\") " pod="openshift-network-operator/iptables-alerter-fwtq4" Sep 10 02:49:03.617764 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.616971 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/936410ac-13a7-43ca-a51f-b0bd96649ff6-host\") pod \"node-ca-mnpps\" (UID: \"936410ac-13a7-43ca-a51f-b0bd96649ff6\") " pod="openshift-image-registry/node-ca-mnpps" Sep 10 02:49:03.617764 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.617172 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d2635452-ab19-4377-93d4-3db8d6e0aa88-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6d6zf\" (UID: \"d2635452-ab19-4377-93d4-3db8d6e0aa88\") " pod="openshift-multus/multus-additional-cni-plugins-6d6zf" Sep 10 02:49:03.617764 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.617644 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/078d27a8-8e62-4092-943b-ade84d558613-iptables-alerter-script\") pod \"iptables-alerter-fwtq4\" (UID: \"078d27a8-8e62-4092-943b-ade84d558613\") " pod="openshift-network-operator/iptables-alerter-fwtq4" Sep 10 02:49:03.618114 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.617879 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d2635452-ab19-4377-93d4-3db8d6e0aa88-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6d6zf\" (UID: \"d2635452-ab19-4377-93d4-3db8d6e0aa88\") " pod="openshift-multus/multus-additional-cni-plugins-6d6zf" Sep 10 02:49:03.618114 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.617900 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/936410ac-13a7-43ca-a51f-b0bd96649ff6-serviceca\") pod \"node-ca-mnpps\" (UID: \"936410ac-13a7-43ca-a51f-b0bd96649ff6\") " pod="openshift-image-registry/node-ca-mnpps" Sep 10 02:49:03.618114 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.617903 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d2635452-ab19-4377-93d4-3db8d6e0aa88-cni-binary-copy\") pod \"multus-additional-cni-plugins-6d6zf\" (UID: \"d2635452-ab19-4377-93d4-3db8d6e0aa88\") " pod="openshift-multus/multus-additional-cni-plugins-6d6zf" Sep 10 02:49:03.618114 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.617963 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d2635452-ab19-4377-93d4-3db8d6e0aa88-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6d6zf\" (UID: \"d2635452-ab19-4377-93d4-3db8d6e0aa88\") " pod="openshift-multus/multus-additional-cni-plugins-6d6zf" Sep 10 02:49:03.627294 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.627271 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-kpzbd" Sep 10 02:49:03.644775 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.644296 2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 10 02:49:03.648938 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.648858 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mr5s\" (UniqueName: \"kubernetes.io/projected/078d27a8-8e62-4092-943b-ade84d558613-kube-api-access-5mr5s\") pod \"iptables-alerter-fwtq4\" (UID: \"078d27a8-8e62-4092-943b-ade84d558613\") " pod="openshift-network-operator/iptables-alerter-fwtq4" Sep 10 02:49:03.648938 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.648928 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fj8jr\" (UniqueName: \"kubernetes.io/projected/d2635452-ab19-4377-93d4-3db8d6e0aa88-kube-api-access-fj8jr\") pod \"multus-additional-cni-plugins-6d6zf\" (UID: \"d2635452-ab19-4377-93d4-3db8d6e0aa88\") " pod="openshift-multus/multus-additional-cni-plugins-6d6zf" Sep 10 02:49:03.651257 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.651229 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbrm7\" (UniqueName: \"kubernetes.io/projected/936410ac-13a7-43ca-a51f-b0bd96649ff6-kube-api-access-tbrm7\") pod \"node-ca-mnpps\" (UID: \"936410ac-13a7-43ca-a51f-b0bd96649ff6\") " pod="openshift-image-registry/node-ca-mnpps" Sep 10 02:49:03.654889 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.654643 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-37.ec2.internal" event={"ID":"8da85c75381fab0440e1fd992aacf5e7","Type":"ContainerStarted","Data":"4def1138106d677d7bacdbd1da5e3150d6f4384c8fa06488dfcfb1770fc3ac9a"} Sep 10 02:49:03.656505 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.656476 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-37.ec2.internal" event={"ID":"480faa79f347b524a5131c7d80428e3a","Type":"ContainerStarted","Data":"de6a730bb61fef1a0d0bffcf1e564d8131d1317501b2f28cd9ae5d67e0437d8c"} Sep 10 02:49:03.717342 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.717253 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c0f2ef31-35db-4d61-b0c4-77358e82154f-etc-sysctl-conf\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.717342 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.717305 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drnkg\" (UniqueName: \"kubernetes.io/projected/c0f2ef31-35db-4d61-b0c4-77358e82154f-kube-api-access-drnkg\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.717342 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.717333 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8c69b77f-74ff-4667-9c26-9beba2e52e21-multus-daemon-config\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.717691 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.717363 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-host-run-netns\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.717691 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.717391 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ff1e29e-7033-490c-9316-def0076b8f7e-original-pull-secret\") pod \"global-pull-secret-syncer-sdxng\" (UID: \"9ff1e29e-7033-490c-9316-def0076b8f7e\") " pod="kube-system/global-pull-secret-syncer-sdxng" Sep 10 02:49:03.717691 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.717441 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-run-ovn\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.717691 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.717469 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8a6680c4-21b8-4fe2-a763-e84858c7ef90-ovnkube-script-lib\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.717691 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.717474 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c0f2ef31-35db-4d61-b0c4-77358e82154f-etc-sysctl-conf\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.717691 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.717491 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tz692\" (UniqueName: \"kubernetes.io/projected/8a6680c4-21b8-4fe2-a763-e84858c7ef90-kube-api-access-tz692\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.717691 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.717556 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-host-run-netns\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.717691 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.717572 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c0f2ef31-35db-4d61-b0c4-77358e82154f-etc-kubernetes\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.717691 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.717601 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8shw\" (UniqueName: \"kubernetes.io/projected/8c69b77f-74ff-4667-9c26-9beba2e52e21-kube-api-access-l8shw\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.717691 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.717625 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8a6680c4-21b8-4fe2-a763-e84858c7ef90-ovnkube-config\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.717691 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.717653 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fdc2579-93fb-494d-b179-46cf1db9aaa8-tmp-dir\") pod \"node-resolver-kkzdh\" (UID: \"6fdc2579-93fb-494d-b179-46cf1db9aaa8\") " pod="openshift-dns/node-resolver-kkzdh" Sep 10 02:49:03.717691 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.717676 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c0f2ef31-35db-4d61-b0c4-77358e82154f-sys\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.717691 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.717699 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c0f2ef31-35db-4d61-b0c4-77358e82154f-host\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.718123 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.717723 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-cnibin\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.718123 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.717745 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-host-var-lib-cni-multus\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.718123 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.717759 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-host-var-lib-kubelet\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.718123 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.717785 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c0f2ef31-35db-4d61-b0c4-77358e82154f-etc-systemd\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.718123 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.717804 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c0f2ef31-35db-4d61-b0c4-77358e82154f-etc-modprobe-d\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.718123 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.717834 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-host-run-k8s-cni-cncf-io\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.718123 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.717857 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-host-run-netns\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.718123 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.717874 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.718123 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.717892 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e-metrics-certs\") pod \"network-metrics-daemon-d2sbz\" (UID: \"c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e\") " pod="openshift-multus/network-metrics-daemon-d2sbz" Sep 10 02:49:03.718123 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.717907 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c0f2ef31-35db-4d61-b0c4-77358e82154f-run\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.718123 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.717922 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9ff1e29e-7033-490c-9316-def0076b8f7e-kubelet-config\") pod \"global-pull-secret-syncer-sdxng\" (UID: \"9ff1e29e-7033-490c-9316-def0076b8f7e\") " pod="kube-system/global-pull-secret-syncer-sdxng" Sep 10 02:49:03.718123 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.717936 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-multus-cni-dir\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.718123 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.717951 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-host-run-ovn-kubernetes\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.718123 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.717966 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c0f2ef31-35db-4d61-b0c4-77358e82154f-var-lib-kubelet\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.718123 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.717982 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8c69b77f-74ff-4667-9c26-9beba2e52e21-cni-binary-copy\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.718123 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.717999 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-systemd-units\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.718123 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718014 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-var-lib-openvswitch\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.718670 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718042 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-log-socket\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.718670 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718061 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c0f2ef31-35db-4d61-b0c4-77358e82154f-etc-sysconfig\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.718670 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718093 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt95b\" (UniqueName: \"kubernetes.io/projected/690238d6-9e65-42b8-b042-d1c3efbc1e45-kube-api-access-pt95b\") pod \"network-check-target-fqfck\" (UID: \"690238d6-9e65-42b8-b042-d1c3efbc1e45\") " pod="openshift-network-diagnostics/network-check-target-fqfck" Sep 10 02:49:03.718670 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718114 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-node-log\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.718670 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718134 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/76f77578-08ad-40d2-849d-7d71c6b7d76b-socket-dir\") pod \"aws-ebs-csi-driver-node-p6kp7\" (UID: \"76f77578-08ad-40d2-849d-7d71c6b7d76b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p6kp7" Sep 10 02:49:03.718670 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718152 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-host-slash\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.718670 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718168 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c0f2ef31-35db-4d61-b0c4-77358e82154f-etc-sysctl-d\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.718670 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718183 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c0f2ef31-35db-4d61-b0c4-77358e82154f-tmp\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.718670 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718198 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-multus-conf-dir\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.718670 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718213 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-run-openvswitch\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.718670 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718224 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8c69b77f-74ff-4667-9c26-9beba2e52e21-multus-daemon-config\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.718670 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718228 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/76f77578-08ad-40d2-849d-7d71c6b7d76b-device-dir\") pod \"aws-ebs-csi-driver-node-p6kp7\" (UID: \"76f77578-08ad-40d2-849d-7d71c6b7d76b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p6kp7" Sep 10 02:49:03.718670 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718273 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9ff1e29e-7033-490c-9316-def0076b8f7e-dbus\") pod \"global-pull-secret-syncer-sdxng\" (UID: \"9ff1e29e-7033-490c-9316-def0076b8f7e\") " pod="kube-system/global-pull-secret-syncer-sdxng" Sep 10 02:49:03.718670 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718276 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/76f77578-08ad-40d2-849d-7d71c6b7d76b-device-dir\") pod \"aws-ebs-csi-driver-node-p6kp7\" (UID: \"76f77578-08ad-40d2-849d-7d71c6b7d76b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p6kp7" Sep 10 02:49:03.718670 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718291 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-system-cni-dir\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.718670 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718309 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-etc-kubernetes\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.718670 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718319 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-run-ovn\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.718670 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718325 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-host-kubelet\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.719434 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718342 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/76f77578-08ad-40d2-849d-7d71c6b7d76b-registration-dir\") pod \"aws-ebs-csi-driver-node-p6kp7\" (UID: \"76f77578-08ad-40d2-849d-7d71c6b7d76b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p6kp7" Sep 10 02:49:03.719434 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718361 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7r8h\" (UniqueName: \"kubernetes.io/projected/76f77578-08ad-40d2-849d-7d71c6b7d76b-kube-api-access-v7r8h\") pod \"aws-ebs-csi-driver-node-p6kp7\" (UID: \"76f77578-08ad-40d2-849d-7d71c6b7d76b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p6kp7" Sep 10 02:49:03.719434 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718378 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-hostroot\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.719434 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718397 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-run-systemd\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.719434 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718414 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/76f77578-08ad-40d2-849d-7d71c6b7d76b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p6kp7\" (UID: \"76f77578-08ad-40d2-849d-7d71c6b7d76b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p6kp7" Sep 10 02:49:03.719434 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718432 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/76f77578-08ad-40d2-849d-7d71c6b7d76b-sys-fs\") pod \"aws-ebs-csi-driver-node-p6kp7\" (UID: \"76f77578-08ad-40d2-849d-7d71c6b7d76b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p6kp7" Sep 10 02:49:03.719434 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718450 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c0f2ef31-35db-4d61-b0c4-77358e82154f-lib-modules\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.719434 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718488 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-multus-socket-dir-parent\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.719434 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718504 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-host-var-lib-cni-bin\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.719434 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718521 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-etc-openvswitch\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.719434 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718538 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6fdc2579-93fb-494d-b179-46cf1db9aaa8-hosts-file\") pod \"node-resolver-kkzdh\" (UID: \"6fdc2579-93fb-494d-b179-46cf1db9aaa8\") " pod="openshift-dns/node-resolver-kkzdh" Sep 10 02:49:03.719434 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718555 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5bhr\" (UniqueName: \"kubernetes.io/projected/6fdc2579-93fb-494d-b179-46cf1db9aaa8-kube-api-access-x5bhr\") pod \"node-resolver-kkzdh\" (UID: \"6fdc2579-93fb-494d-b179-46cf1db9aaa8\") " pod="openshift-dns/node-resolver-kkzdh" Sep 10 02:49:03.719434 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718570 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-host-run-multus-certs\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.719434 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718603 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-host-cni-bin\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.719434 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718621 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8a6680c4-21b8-4fe2-a763-e84858c7ef90-env-overrides\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.719434 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718648 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8a6680c4-21b8-4fe2-a763-e84858c7ef90-ovn-node-metrics-cert\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.719434 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718668 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/27be8319-a821-4e27-9dc8-411257676cb7-agent-certs\") pod \"konnectivity-agent-z9d9b\" (UID: \"27be8319-a821-4e27-9dc8-411257676cb7\") " pod="kube-system/konnectivity-agent-z9d9b" Sep 10 02:49:03.720269 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718689 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c0f2ef31-35db-4d61-b0c4-77358e82154f-etc-tuned\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.720269 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718715 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/27be8319-a821-4e27-9dc8-411257676cb7-konnectivity-ca\") pod \"konnectivity-agent-z9d9b\" (UID: \"27be8319-a821-4e27-9dc8-411257676cb7\") " pod="kube-system/konnectivity-agent-z9d9b" Sep 10 02:49:03.720269 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718743 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/76f77578-08ad-40d2-849d-7d71c6b7d76b-etc-selinux\") pod \"aws-ebs-csi-driver-node-p6kp7\" (UID: \"76f77578-08ad-40d2-849d-7d71c6b7d76b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p6kp7" Sep 10 02:49:03.720269 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718772 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smjcj\" (UniqueName: \"kubernetes.io/projected/c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e-kube-api-access-smjcj\") pod \"network-metrics-daemon-d2sbz\" (UID: \"c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e\") " pod="openshift-multus/network-metrics-daemon-d2sbz" Sep 10 02:49:03.720269 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718809 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-host-cni-netd\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.720269 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718822 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8a6680c4-21b8-4fe2-a763-e84858c7ef90-ovnkube-script-lib\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.720269 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718890 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-host-var-lib-cni-multus\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.720269 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718926 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-cnibin\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.720269 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718944 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-host-var-lib-kubelet\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.720269 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718983 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c0f2ef31-35db-4d61-b0c4-77358e82154f-etc-systemd\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.720269 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.718981 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9ff1e29e-7033-490c-9316-def0076b8f7e-dbus\") pod \"global-pull-secret-syncer-sdxng\" (UID: \"9ff1e29e-7033-490c-9316-def0076b8f7e\") " pod="kube-system/global-pull-secret-syncer-sdxng" Sep 10 02:49:03.720269 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.719072 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c0f2ef31-35db-4d61-b0c4-77358e82154f-etc-modprobe-d\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.720269 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.719080 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-system-cni-dir\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.720269 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.719110 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-host-run-k8s-cni-cncf-io\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.720269 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.719154 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-host-run-netns\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.720269 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:03.719166 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 02:49:03.720269 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.719201 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.721207 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:03.719255 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ff1e29e-7033-490c-9316-def0076b8f7e-original-pull-secret podName:9ff1e29e-7033-490c-9316-def0076b8f7e nodeName:}" failed. No retries permitted until 2026-09-10 02:49:04.219215059 +0000 UTC m=+3.066044327 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9ff1e29e-7033-490c-9316-def0076b8f7e-original-pull-secret") pod "global-pull-secret-syncer-sdxng" (UID: "9ff1e29e-7033-490c-9316-def0076b8f7e") : object "kube-system"/"original-pull-secret" not registered Sep 10 02:49:03.721207 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.719266 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-host-cni-bin\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.721207 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:03.719300 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 02:49:03.721207 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.719320 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-etc-kubernetes\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.721207 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:03.719349 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e-metrics-certs podName:c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e nodeName:}" failed. No retries permitted until 2026-09-10 02:49:04.219333504 +0000 UTC m=+3.066162758 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e-metrics-certs") pod "network-metrics-daemon-d2sbz" (UID: "c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 02:49:03.721207 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.719360 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-host-kubelet\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.721207 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.719415 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-systemd-units\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.721207 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.719429 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-run-systemd\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.721207 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.719453 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-hostroot\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.721207 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.719481 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/76f77578-08ad-40d2-849d-7d71c6b7d76b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p6kp7\" (UID: \"76f77578-08ad-40d2-849d-7d71c6b7d76b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p6kp7" Sep 10 02:49:03.721207 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.719552 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/76f77578-08ad-40d2-849d-7d71c6b7d76b-sys-fs\") pod \"aws-ebs-csi-driver-node-p6kp7\" (UID: \"76f77578-08ad-40d2-849d-7d71c6b7d76b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p6kp7" Sep 10 02:49:03.721207 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.719643 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c0f2ef31-35db-4d61-b0c4-77358e82154f-lib-modules\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.721207 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.719791 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-multus-socket-dir-parent\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.721207 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.719826 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8a6680c4-21b8-4fe2-a763-e84858c7ef90-env-overrides\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.721207 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.719840 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-host-var-lib-cni-bin\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.721207 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.719872 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8c69b77f-74ff-4667-9c26-9beba2e52e21-cni-binary-copy\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.721207 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.719948 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/76f77578-08ad-40d2-849d-7d71c6b7d76b-registration-dir\") pod \"aws-ebs-csi-driver-node-p6kp7\" (UID: \"76f77578-08ad-40d2-849d-7d71c6b7d76b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p6kp7" Sep 10 02:49:03.721893 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.720013 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-etc-openvswitch\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.721893 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.720041 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6fdc2579-93fb-494d-b179-46cf1db9aaa8-hosts-file\") pod \"node-resolver-kkzdh\" (UID: \"6fdc2579-93fb-494d-b179-46cf1db9aaa8\") " pod="openshift-dns/node-resolver-kkzdh" Sep 10 02:49:03.721893 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.720184 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-multus-cni-dir\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.721893 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.720227 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-host-run-multus-certs\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.721893 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.720229 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c0f2ef31-35db-4d61-b0c4-77358e82154f-etc-sysctl-d\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.721893 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.720273 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9ff1e29e-7033-490c-9316-def0076b8f7e-kubelet-config\") pod \"global-pull-secret-syncer-sdxng\" (UID: \"9ff1e29e-7033-490c-9316-def0076b8f7e\") " pod="kube-system/global-pull-secret-syncer-sdxng" Sep 10 02:49:03.721893 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.720278 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-host-run-ovn-kubernetes\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.721893 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.720287 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c0f2ef31-35db-4d61-b0c4-77358e82154f-run\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.721893 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.720312 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8c69b77f-74ff-4667-9c26-9beba2e52e21-multus-conf-dir\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.721893 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.720331 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c0f2ef31-35db-4d61-b0c4-77358e82154f-host\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.721893 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.720336 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/76f77578-08ad-40d2-849d-7d71c6b7d76b-etc-selinux\") pod \"aws-ebs-csi-driver-node-p6kp7\" (UID: \"76f77578-08ad-40d2-849d-7d71c6b7d76b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p6kp7" Sep 10 02:49:03.721893 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.720363 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-var-lib-openvswitch\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.721893 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.720375 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-run-openvswitch\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.721893 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.720406 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-log-socket\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.721893 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.720461 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c0f2ef31-35db-4d61-b0c4-77358e82154f-etc-sysconfig\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.721893 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.720567 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-host-cni-netd\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.721893 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.720640 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c0f2ef31-35db-4d61-b0c4-77358e82154f-var-lib-kubelet\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.721893 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.720683 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-node-log\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.722788 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.720827 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c0f2ef31-35db-4d61-b0c4-77358e82154f-sys\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.722788 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.720840 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8a6680c4-21b8-4fe2-a763-e84858c7ef90-ovnkube-config\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.722788 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.720877 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8a6680c4-21b8-4fe2-a763-e84858c7ef90-host-slash\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.722788 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.720944 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c0f2ef31-35db-4d61-b0c4-77358e82154f-etc-kubernetes\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.722788 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.720949 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/76f77578-08ad-40d2-849d-7d71c6b7d76b-socket-dir\") pod \"aws-ebs-csi-driver-node-p6kp7\" (UID: \"76f77578-08ad-40d2-849d-7d71c6b7d76b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p6kp7" Sep 10 02:49:03.722788 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.721089 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/27be8319-a821-4e27-9dc8-411257676cb7-konnectivity-ca\") pod \"konnectivity-agent-z9d9b\" (UID: \"27be8319-a821-4e27-9dc8-411257676cb7\") " pod="kube-system/konnectivity-agent-z9d9b" Sep 10 02:49:03.722788 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.721117 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fdc2579-93fb-494d-b179-46cf1db9aaa8-tmp-dir\") pod \"node-resolver-kkzdh\" (UID: \"6fdc2579-93fb-494d-b179-46cf1db9aaa8\") " pod="openshift-dns/node-resolver-kkzdh" Sep 10 02:49:03.723445 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.723420 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8a6680c4-21b8-4fe2-a763-e84858c7ef90-ovn-node-metrics-cert\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.724712 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.724686 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/27be8319-a821-4e27-9dc8-411257676cb7-agent-certs\") pod \"konnectivity-agent-z9d9b\" (UID: \"27be8319-a821-4e27-9dc8-411257676cb7\") " pod="kube-system/konnectivity-agent-z9d9b" Sep 10 02:49:03.726148 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.726105 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c0f2ef31-35db-4d61-b0c4-77358e82154f-tmp\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.727180 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.727157 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c0f2ef31-35db-4d61-b0c4-77358e82154f-etc-tuned\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.727680 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.727655 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drnkg\" (UniqueName: \"kubernetes.io/projected/c0f2ef31-35db-4d61-b0c4-77358e82154f-kube-api-access-drnkg\") pod \"tuned-xbp79\" (UID: \"c0f2ef31-35db-4d61-b0c4-77358e82154f\") " pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.732312 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.732268 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8shw\" (UniqueName: \"kubernetes.io/projected/8c69b77f-74ff-4667-9c26-9beba2e52e21-kube-api-access-l8shw\") pod \"multus-28crk\" (UID: \"8c69b77f-74ff-4667-9c26-9beba2e52e21\") " pod="openshift-multus/multus-28crk" Sep 10 02:49:03.732312 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.732289 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5bhr\" (UniqueName: \"kubernetes.io/projected/6fdc2579-93fb-494d-b179-46cf1db9aaa8-kube-api-access-x5bhr\") pod \"node-resolver-kkzdh\" (UID: \"6fdc2579-93fb-494d-b179-46cf1db9aaa8\") " pod="openshift-dns/node-resolver-kkzdh" Sep 10 02:49:03.733067 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.733012 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smjcj\" (UniqueName: \"kubernetes.io/projected/c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e-kube-api-access-smjcj\") pod \"network-metrics-daemon-d2sbz\" (UID: \"c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e\") " pod="openshift-multus/network-metrics-daemon-d2sbz" Sep 10 02:49:03.734606 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:03.734570 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 02:49:03.734606 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:03.734603 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 02:49:03.734764 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:03.734620 2560 projected.go:196] Error preparing data for projected volume kube-api-access-pt95b for pod openshift-network-diagnostics/network-check-target-fqfck: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 02:49:03.734764 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:03.734697 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/690238d6-9e65-42b8-b042-d1c3efbc1e45-kube-api-access-pt95b podName:690238d6-9e65-42b8-b042-d1c3efbc1e45 nodeName:}" failed. No retries permitted until 2026-09-10 02:49:04.23467731 +0000 UTC m=+3.081506780 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-pt95b" (UniqueName: "kubernetes.io/projected/690238d6-9e65-42b8-b042-d1c3efbc1e45-kube-api-access-pt95b") pod "network-check-target-fqfck" (UID: "690238d6-9e65-42b8-b042-d1c3efbc1e45") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 02:49:03.736101 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.736070 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7r8h\" (UniqueName: \"kubernetes.io/projected/76f77578-08ad-40d2-849d-7d71c6b7d76b-kube-api-access-v7r8h\") pod \"aws-ebs-csi-driver-node-p6kp7\" (UID: \"76f77578-08ad-40d2-849d-7d71c6b7d76b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p6kp7" Sep 10 02:49:03.736476 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.736458 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tz692\" (UniqueName: \"kubernetes.io/projected/8a6680c4-21b8-4fe2-a763-e84858c7ef90-kube-api-access-tz692\") pod \"ovnkube-node-swhbm\" (UID: \"8a6680c4-21b8-4fe2-a763-e84858c7ef90\") " pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.815782 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.815728 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fwtq4" Sep 10 02:49:03.828895 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:03.828866 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod078d27a8_8e62_4092_943b_ade84d558613.slice/crio-907442b715ae76c77517f9abae0f2aa54d3ff5533c63d0c3180b126fe4fdd092 WatchSource:0}: Error finding container 907442b715ae76c77517f9abae0f2aa54d3ff5533c63d0c3180b126fe4fdd092: Status 404 returned error can't find the container with id 907442b715ae76c77517f9abae0f2aa54d3ff5533c63d0c3180b126fe4fdd092 Sep 10 02:49:03.831099 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.831030 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mnpps" Sep 10 02:49:03.842392 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:03.842362 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod936410ac_13a7_43ca_a51f_b0bd96649ff6.slice/crio-10bb3eae8caa4da018e836bf06f9c94cb4028f6ef8e7f30751bdcb1ac2698df4 WatchSource:0}: Error finding container 10bb3eae8caa4da018e836bf06f9c94cb4028f6ef8e7f30751bdcb1ac2698df4: Status 404 returned error can't find the container with id 10bb3eae8caa4da018e836bf06f9c94cb4028f6ef8e7f30751bdcb1ac2698df4 Sep 10 02:49:03.844134 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.843990 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6d6zf" Sep 10 02:49:03.850011 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.849822 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:03.854213 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:03.854184 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2635452_ab19_4377_93d4_3db8d6e0aa88.slice/crio-3f1cef7b1784e73b70d313aa62ee869626a8d8b263a22be1b3227f3f4653c69b WatchSource:0}: Error finding container 3f1cef7b1784e73b70d313aa62ee869626a8d8b263a22be1b3227f3f4653c69b: Status 404 returned error can't find the container with id 3f1cef7b1784e73b70d313aa62ee869626a8d8b263a22be1b3227f3f4653c69b Sep 10 02:49:03.859009 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.858985 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-z9d9b" Sep 10 02:49:03.859724 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:03.859689 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a6680c4_21b8_4fe2_a763_e84858c7ef90.slice/crio-e442b36482013b04969ae45bb3b28c9b96c62abc6746db0e33937af67091c45d WatchSource:0}: Error finding container e442b36482013b04969ae45bb3b28c9b96c62abc6746db0e33937af67091c45d: Status 404 returned error can't find the container with id e442b36482013b04969ae45bb3b28c9b96c62abc6746db0e33937af67091c45d Sep 10 02:49:03.869966 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.867297 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kkzdh" Sep 10 02:49:03.871607 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:03.871028 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27be8319_a821_4e27_9dc8_411257676cb7.slice/crio-44cebf45f46a9b03397b38dcf83840b2811dd2a24fec09d8d9b704aa7647f497 WatchSource:0}: Error finding container 44cebf45f46a9b03397b38dcf83840b2811dd2a24fec09d8d9b704aa7647f497: Status 404 returned error can't find the container with id 44cebf45f46a9b03397b38dcf83840b2811dd2a24fec09d8d9b704aa7647f497 Sep 10 02:49:03.876118 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.876093 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p6kp7" Sep 10 02:49:03.877795 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:03.877768 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fdc2579_93fb_494d_b179_46cf1db9aaa8.slice/crio-2bd1433b070b453eef0a2658afbef147beb724f9d7861e566ef5849b107466af WatchSource:0}: Error finding container 2bd1433b070b453eef0a2658afbef147beb724f9d7861e566ef5849b107466af: Status 404 returned error can't find the container with id 2bd1433b070b453eef0a2658afbef147beb724f9d7861e566ef5849b107466af Sep 10 02:49:03.884661 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.884639 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xbp79" Sep 10 02:49:03.887695 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:03.887665 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76f77578_08ad_40d2_849d_7d71c6b7d76b.slice/crio-dac6953435b0833c698ad7848e41fd24b40cd12d37697c88e232a72c27e91208 WatchSource:0}: Error finding container dac6953435b0833c698ad7848e41fd24b40cd12d37697c88e232a72c27e91208: Status 404 returned error can't find the container with id dac6953435b0833c698ad7848e41fd24b40cd12d37697c88e232a72c27e91208 Sep 10 02:49:03.896637 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:03.896601 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0f2ef31_35db_4d61_b0c4_77358e82154f.slice/crio-6d272cbaa9f0ea9f4502ac848070cd925fa40d40fca78ed59c271364a4759607 WatchSource:0}: Error finding container 6d272cbaa9f0ea9f4502ac848070cd925fa40d40fca78ed59c271364a4759607: Status 404 returned error can't find the container with id 6d272cbaa9f0ea9f4502ac848070cd925fa40d40fca78ed59c271364a4759607 Sep 10 02:49:03.898732 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:03.898712 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-28crk" Sep 10 02:49:03.910737 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:03.910691 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c69b77f_74ff_4667_9c26_9beba2e52e21.slice/crio-2035c801aca0e604765570f6911a147f50cf5691c5814f5b0c7144849ba6b90b WatchSource:0}: Error finding container 2035c801aca0e604765570f6911a147f50cf5691c5814f5b0c7144849ba6b90b: Status 404 returned error can't find the container with id 2035c801aca0e604765570f6911a147f50cf5691c5814f5b0c7144849ba6b90b Sep 10 02:49:04.222828 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:04.222796 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e-metrics-certs\") pod \"network-metrics-daemon-d2sbz\" (UID: \"c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e\") " pod="openshift-multus/network-metrics-daemon-d2sbz" Sep 10 02:49:04.223047 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:04.222885 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ff1e29e-7033-490c-9316-def0076b8f7e-original-pull-secret\") pod \"global-pull-secret-syncer-sdxng\" (UID: \"9ff1e29e-7033-490c-9316-def0076b8f7e\") " pod="kube-system/global-pull-secret-syncer-sdxng" Sep 10 02:49:04.223047 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:04.222981 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 02:49:04.223047 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:04.223042 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 02:49:04.223212 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:04.223064 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e-metrics-certs podName:c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e nodeName:}" failed. No retries permitted until 2026-09-10 02:49:05.223044437 +0000 UTC m=+4.069873707 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e-metrics-certs") pod "network-metrics-daemon-d2sbz" (UID: "c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 02:49:04.223212 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:04.223109 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ff1e29e-7033-490c-9316-def0076b8f7e-original-pull-secret podName:9ff1e29e-7033-490c-9316-def0076b8f7e nodeName:}" failed. No retries permitted until 2026-09-10 02:49:05.223091919 +0000 UTC m=+4.069921169 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9ff1e29e-7033-490c-9316-def0076b8f7e-original-pull-secret") pod "global-pull-secret-syncer-sdxng" (UID: "9ff1e29e-7033-490c-9316-def0076b8f7e") : object "kube-system"/"original-pull-secret" not registered Sep 10 02:49:04.323850 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:04.323816 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt95b\" (UniqueName: \"kubernetes.io/projected/690238d6-9e65-42b8-b042-d1c3efbc1e45-kube-api-access-pt95b\") pod \"network-check-target-fqfck\" (UID: \"690238d6-9e65-42b8-b042-d1c3efbc1e45\") " pod="openshift-network-diagnostics/network-check-target-fqfck" Sep 10 02:49:04.324021 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:04.323998 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 02:49:04.324021 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:04.324018 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 02:49:04.324146 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:04.324029 2560 projected.go:196] Error preparing data for projected volume kube-api-access-pt95b for pod openshift-network-diagnostics/network-check-target-fqfck: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 02:49:04.324146 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:04.324093 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/690238d6-9e65-42b8-b042-d1c3efbc1e45-kube-api-access-pt95b podName:690238d6-9e65-42b8-b042-d1c3efbc1e45 nodeName:}" failed. No retries permitted until 2026-09-10 02:49:05.324076579 +0000 UTC m=+4.170905833 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-pt95b" (UniqueName: "kubernetes.io/projected/690238d6-9e65-42b8-b042-d1c3efbc1e45-kube-api-access-pt95b") pod "network-check-target-fqfck" (UID: "690238d6-9e65-42b8-b042-d1c3efbc1e45") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 02:49:04.633063 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:04.632938 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 02:44:04 +0000 UTC" deadline="2028-03-26 00:59:50.147273474 +0000 UTC" Sep 10 02:49:04.633063 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:04.633007 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13510h10m45.514270629s" Sep 10 02:49:04.659808 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:04.659770 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p6kp7" event={"ID":"76f77578-08ad-40d2-849d-7d71c6b7d76b","Type":"ContainerStarted","Data":"dac6953435b0833c698ad7848e41fd24b40cd12d37697c88e232a72c27e91208"} Sep 10 02:49:04.660925 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:04.660901 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kkzdh" event={"ID":"6fdc2579-93fb-494d-b179-46cf1db9aaa8","Type":"ContainerStarted","Data":"2bd1433b070b453eef0a2658afbef147beb724f9d7861e566ef5849b107466af"} Sep 10 02:49:04.662056 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:04.662028 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" event={"ID":"8a6680c4-21b8-4fe2-a763-e84858c7ef90","Type":"ContainerStarted","Data":"e442b36482013b04969ae45bb3b28c9b96c62abc6746db0e33937af67091c45d"} Sep 10 02:49:04.663053 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:04.663031 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mnpps" event={"ID":"936410ac-13a7-43ca-a51f-b0bd96649ff6","Type":"ContainerStarted","Data":"10bb3eae8caa4da018e836bf06f9c94cb4028f6ef8e7f30751bdcb1ac2698df4"} Sep 10 02:49:04.664036 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:04.664008 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fwtq4" event={"ID":"078d27a8-8e62-4092-943b-ade84d558613","Type":"ContainerStarted","Data":"907442b715ae76c77517f9abae0f2aa54d3ff5533c63d0c3180b126fe4fdd092"} Sep 10 02:49:04.665288 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:04.665267 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-28crk" event={"ID":"8c69b77f-74ff-4667-9c26-9beba2e52e21","Type":"ContainerStarted","Data":"2035c801aca0e604765570f6911a147f50cf5691c5814f5b0c7144849ba6b90b"} Sep 10 02:49:04.666260 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:04.666238 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xbp79" event={"ID":"c0f2ef31-35db-4d61-b0c4-77358e82154f","Type":"ContainerStarted","Data":"6d272cbaa9f0ea9f4502ac848070cd925fa40d40fca78ed59c271364a4759607"} Sep 10 02:49:04.667421 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:04.667391 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-z9d9b" event={"ID":"27be8319-a821-4e27-9dc8-411257676cb7","Type":"ContainerStarted","Data":"44cebf45f46a9b03397b38dcf83840b2811dd2a24fec09d8d9b704aa7647f497"} Sep 10 02:49:04.668644 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:04.668624 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6d6zf" event={"ID":"d2635452-ab19-4377-93d4-3db8d6e0aa88","Type":"ContainerStarted","Data":"3f1cef7b1784e73b70d313aa62ee869626a8d8b263a22be1b3227f3f4653c69b"} Sep 10 02:49:05.232375 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:05.231953 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ff1e29e-7033-490c-9316-def0076b8f7e-original-pull-secret\") pod \"global-pull-secret-syncer-sdxng\" (UID: \"9ff1e29e-7033-490c-9316-def0076b8f7e\") " pod="kube-system/global-pull-secret-syncer-sdxng" Sep 10 02:49:05.232375 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:05.232018 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e-metrics-certs\") pod \"network-metrics-daemon-d2sbz\" (UID: \"c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e\") " pod="openshift-multus/network-metrics-daemon-d2sbz" Sep 10 02:49:05.232375 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:05.232182 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 02:49:05.232375 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:05.232245 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e-metrics-certs podName:c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e nodeName:}" failed. No retries permitted until 2026-09-10 02:49:07.232224006 +0000 UTC m=+6.079053280 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e-metrics-certs") pod "network-metrics-daemon-d2sbz" (UID: "c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 02:49:05.232375 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:05.232312 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 02:49:05.232375 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:05.232347 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ff1e29e-7033-490c-9316-def0076b8f7e-original-pull-secret podName:9ff1e29e-7033-490c-9316-def0076b8f7e nodeName:}" failed. No retries permitted until 2026-09-10 02:49:07.232335727 +0000 UTC m=+6.079164990 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9ff1e29e-7033-490c-9316-def0076b8f7e-original-pull-secret") pod "global-pull-secret-syncer-sdxng" (UID: "9ff1e29e-7033-490c-9316-def0076b8f7e") : object "kube-system"/"original-pull-secret" not registered Sep 10 02:49:05.333320 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:05.332457 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt95b\" (UniqueName: \"kubernetes.io/projected/690238d6-9e65-42b8-b042-d1c3efbc1e45-kube-api-access-pt95b\") pod \"network-check-target-fqfck\" (UID: \"690238d6-9e65-42b8-b042-d1c3efbc1e45\") " pod="openshift-network-diagnostics/network-check-target-fqfck" Sep 10 02:49:05.333320 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:05.332868 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 02:49:05.333320 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:05.332894 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 02:49:05.333320 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:05.332906 2560 projected.go:196] Error preparing data for projected volume kube-api-access-pt95b for pod openshift-network-diagnostics/network-check-target-fqfck: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 02:49:05.333320 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:05.332971 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/690238d6-9e65-42b8-b042-d1c3efbc1e45-kube-api-access-pt95b podName:690238d6-9e65-42b8-b042-d1c3efbc1e45 nodeName:}" failed. No retries permitted until 2026-09-10 02:49:07.332951846 +0000 UTC m=+6.179781109 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-pt95b" (UniqueName: "kubernetes.io/projected/690238d6-9e65-42b8-b042-d1c3efbc1e45-kube-api-access-pt95b") pod "network-check-target-fqfck" (UID: "690238d6-9e65-42b8-b042-d1c3efbc1e45") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 02:49:05.636958 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:05.636695 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 02:44:04 +0000 UTC" deadline="2028-05-23 18:55:16.880095874 +0000 UTC" Sep 10 02:49:05.636958 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:05.636741 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14920h6m11.243360692s" Sep 10 02:49:05.642903 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:05.642281 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2sbz" Sep 10 02:49:05.642903 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:05.642429 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d2sbz" podUID="c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e" Sep 10 02:49:05.642903 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:05.642875 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sdxng" Sep 10 02:49:05.643171 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:05.642984 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sdxng" podUID="9ff1e29e-7033-490c-9316-def0076b8f7e" Sep 10 02:49:05.643171 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:05.643048 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fqfck" Sep 10 02:49:05.643171 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:05.643118 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fqfck" podUID="690238d6-9e65-42b8-b042-d1c3efbc1e45" Sep 10 02:49:05.692870 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:05.692829 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-37.ec2.internal" event={"ID":"480faa79f347b524a5131c7d80428e3a","Type":"ContainerStarted","Data":"4a421fdef44468c7b12f922de4741d9cf9d0eac7267cf17e1d39121dde10a7e6"} Sep 10 02:49:06.703086 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:06.701710 2560 generic.go:356] "Generic (PLEG): container finished" podID="8da85c75381fab0440e1fd992aacf5e7" containerID="6d8eb71d74f676c1d7d46858cdf0689fd96875374244a9e823ac4a05816e1958" exitCode=0 Sep 10 02:49:06.703086 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:06.702764 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-37.ec2.internal" event={"ID":"8da85c75381fab0440e1fd992aacf5e7","Type":"ContainerDied","Data":"6d8eb71d74f676c1d7d46858cdf0689fd96875374244a9e823ac4a05816e1958"} Sep 10 02:49:06.729111 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:06.728204 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-37.ec2.internal" podStartSLOduration=3.728179733 podStartE2EDuration="3.728179733s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 02:49:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:49:05.720261817 +0000 UTC m=+4.567091093" watchObservedRunningTime="2026-09-10 02:49:06.728179733 +0000 UTC m=+5.575009006" Sep 10 02:49:07.255773 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:07.254904 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e-metrics-certs\") pod \"network-metrics-daemon-d2sbz\" (UID: \"c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e\") " pod="openshift-multus/network-metrics-daemon-d2sbz" Sep 10 02:49:07.255773 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:07.255000 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ff1e29e-7033-490c-9316-def0076b8f7e-original-pull-secret\") pod \"global-pull-secret-syncer-sdxng\" (UID: \"9ff1e29e-7033-490c-9316-def0076b8f7e\") " pod="kube-system/global-pull-secret-syncer-sdxng" Sep 10 02:49:07.255773 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:07.255130 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 02:49:07.255773 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:07.255188 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ff1e29e-7033-490c-9316-def0076b8f7e-original-pull-secret podName:9ff1e29e-7033-490c-9316-def0076b8f7e nodeName:}" failed. No retries permitted until 2026-09-10 02:49:11.255174027 +0000 UTC m=+10.102003274 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9ff1e29e-7033-490c-9316-def0076b8f7e-original-pull-secret") pod "global-pull-secret-syncer-sdxng" (UID: "9ff1e29e-7033-490c-9316-def0076b8f7e") : object "kube-system"/"original-pull-secret" not registered Sep 10 02:49:07.255773 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:07.255529 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 02:49:07.255773 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:07.255567 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e-metrics-certs podName:c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e nodeName:}" failed. No retries permitted until 2026-09-10 02:49:11.255555288 +0000 UTC m=+10.102384535 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e-metrics-certs") pod "network-metrics-daemon-d2sbz" (UID: "c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 02:49:07.356193 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:07.356111 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt95b\" (UniqueName: \"kubernetes.io/projected/690238d6-9e65-42b8-b042-d1c3efbc1e45-kube-api-access-pt95b\") pod \"network-check-target-fqfck\" (UID: \"690238d6-9e65-42b8-b042-d1c3efbc1e45\") " pod="openshift-network-diagnostics/network-check-target-fqfck" Sep 10 02:49:07.356367 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:07.356312 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 02:49:07.356367 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:07.356333 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 02:49:07.356367 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:07.356347 2560 projected.go:196] Error preparing data for projected volume kube-api-access-pt95b for pod openshift-network-diagnostics/network-check-target-fqfck: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 02:49:07.356486 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:07.356414 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/690238d6-9e65-42b8-b042-d1c3efbc1e45-kube-api-access-pt95b podName:690238d6-9e65-42b8-b042-d1c3efbc1e45 nodeName:}" failed. No retries permitted until 2026-09-10 02:49:11.35639305 +0000 UTC m=+10.203222301 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-pt95b" (UniqueName: "kubernetes.io/projected/690238d6-9e65-42b8-b042-d1c3efbc1e45-kube-api-access-pt95b") pod "network-check-target-fqfck" (UID: "690238d6-9e65-42b8-b042-d1c3efbc1e45") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 02:49:07.640811 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:07.640713 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sdxng" Sep 10 02:49:07.640990 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:07.640713 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2sbz" Sep 10 02:49:07.640990 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:07.640860 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sdxng" podUID="9ff1e29e-7033-490c-9316-def0076b8f7e" Sep 10 02:49:07.640990 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:07.640886 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fqfck" Sep 10 02:49:07.641145 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:07.641002 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d2sbz" podUID="c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e" Sep 10 02:49:07.641145 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:07.641086 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fqfck" podUID="690238d6-9e65-42b8-b042-d1c3efbc1e45" Sep 10 02:49:07.707569 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:07.707368 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-37.ec2.internal" event={"ID":"8da85c75381fab0440e1fd992aacf5e7","Type":"ContainerStarted","Data":"61494dc37460b960e9ffd344e45177e55779ea27daf3a05029e22c8f6cb9cd7d"} Sep 10 02:49:09.642618 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:09.642565 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2sbz" Sep 10 02:49:09.643041 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:09.642740 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d2sbz" podUID="c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e" Sep 10 02:49:09.643041 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:09.642796 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sdxng" Sep 10 02:49:09.643041 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:09.642887 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sdxng" podUID="9ff1e29e-7033-490c-9316-def0076b8f7e" Sep 10 02:49:09.643041 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:09.642926 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fqfck" Sep 10 02:49:09.643041 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:09.642989 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fqfck" podUID="690238d6-9e65-42b8-b042-d1c3efbc1e45" Sep 10 02:49:11.291821 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:11.291766 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ff1e29e-7033-490c-9316-def0076b8f7e-original-pull-secret\") pod \"global-pull-secret-syncer-sdxng\" (UID: \"9ff1e29e-7033-490c-9316-def0076b8f7e\") " pod="kube-system/global-pull-secret-syncer-sdxng" Sep 10 02:49:11.292367 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:11.291842 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e-metrics-certs\") pod \"network-metrics-daemon-d2sbz\" (UID: \"c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e\") " pod="openshift-multus/network-metrics-daemon-d2sbz" Sep 10 02:49:11.292367 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:11.292014 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 02:49:11.292367 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:11.292080 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e-metrics-certs podName:c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e nodeName:}" failed. No retries permitted until 2026-09-10 02:49:19.292061274 +0000 UTC m=+18.138890538 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e-metrics-certs") pod "network-metrics-daemon-d2sbz" (UID: "c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 02:49:11.292644 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:11.292482 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 02:49:11.292644 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:11.292537 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ff1e29e-7033-490c-9316-def0076b8f7e-original-pull-secret podName:9ff1e29e-7033-490c-9316-def0076b8f7e nodeName:}" failed. No retries permitted until 2026-09-10 02:49:19.292523154 +0000 UTC m=+18.139352407 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9ff1e29e-7033-490c-9316-def0076b8f7e-original-pull-secret") pod "global-pull-secret-syncer-sdxng" (UID: "9ff1e29e-7033-490c-9316-def0076b8f7e") : object "kube-system"/"original-pull-secret" not registered Sep 10 02:49:11.393485 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:11.392838 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt95b\" (UniqueName: \"kubernetes.io/projected/690238d6-9e65-42b8-b042-d1c3efbc1e45-kube-api-access-pt95b\") pod \"network-check-target-fqfck\" (UID: \"690238d6-9e65-42b8-b042-d1c3efbc1e45\") " pod="openshift-network-diagnostics/network-check-target-fqfck" Sep 10 02:49:11.393485 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:11.393022 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 02:49:11.393485 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:11.393044 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 02:49:11.393485 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:11.393058 2560 projected.go:196] Error preparing data for projected volume kube-api-access-pt95b for pod openshift-network-diagnostics/network-check-target-fqfck: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 02:49:11.393485 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:11.393121 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/690238d6-9e65-42b8-b042-d1c3efbc1e45-kube-api-access-pt95b podName:690238d6-9e65-42b8-b042-d1c3efbc1e45 nodeName:}" failed. No retries permitted until 2026-09-10 02:49:19.393101625 +0000 UTC m=+18.239930886 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-pt95b" (UniqueName: "kubernetes.io/projected/690238d6-9e65-42b8-b042-d1c3efbc1e45-kube-api-access-pt95b") pod "network-check-target-fqfck" (UID: "690238d6-9e65-42b8-b042-d1c3efbc1e45") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 02:49:11.641501 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:11.641417 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2sbz" Sep 10 02:49:11.641652 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:11.641523 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d2sbz" podUID="c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e" Sep 10 02:49:11.641652 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:11.641570 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sdxng" Sep 10 02:49:11.641652 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:11.641621 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fqfck" Sep 10 02:49:11.641820 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:11.641717 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fqfck" podUID="690238d6-9e65-42b8-b042-d1c3efbc1e45" Sep 10 02:49:11.641858 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:11.641807 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sdxng" podUID="9ff1e29e-7033-490c-9316-def0076b8f7e" Sep 10 02:49:13.643058 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:13.640845 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2sbz" Sep 10 02:49:13.643058 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:13.640848 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sdxng" Sep 10 02:49:13.643058 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:13.641017 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fqfck" Sep 10 02:49:13.643058 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:13.641007 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d2sbz" podUID="c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e" Sep 10 02:49:13.643058 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:13.641141 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sdxng" podUID="9ff1e29e-7033-490c-9316-def0076b8f7e" Sep 10 02:49:13.643058 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:13.641227 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fqfck" podUID="690238d6-9e65-42b8-b042-d1c3efbc1e45" Sep 10 02:49:15.640436 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:15.640394 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fqfck" Sep 10 02:49:15.641060 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:15.640394 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2sbz" Sep 10 02:49:15.641060 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:15.640560 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fqfck" podUID="690238d6-9e65-42b8-b042-d1c3efbc1e45" Sep 10 02:49:15.641060 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:15.640660 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d2sbz" podUID="c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e" Sep 10 02:49:15.641060 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:15.640394 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sdxng" Sep 10 02:49:15.641060 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:15.640764 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sdxng" podUID="9ff1e29e-7033-490c-9316-def0076b8f7e" Sep 10 02:49:17.640151 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:17.640112 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fqfck" Sep 10 02:49:17.640608 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:17.640163 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2sbz" Sep 10 02:49:17.640608 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:17.640110 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sdxng" Sep 10 02:49:17.640608 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:17.640235 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fqfck" podUID="690238d6-9e65-42b8-b042-d1c3efbc1e45" Sep 10 02:49:17.640608 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:17.640307 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sdxng" podUID="9ff1e29e-7033-490c-9316-def0076b8f7e" Sep 10 02:49:17.640608 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:17.640396 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d2sbz" podUID="c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e" Sep 10 02:49:19.358087 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:19.358054 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ff1e29e-7033-490c-9316-def0076b8f7e-original-pull-secret\") pod \"global-pull-secret-syncer-sdxng\" (UID: \"9ff1e29e-7033-490c-9316-def0076b8f7e\") " pod="kube-system/global-pull-secret-syncer-sdxng" Sep 10 02:49:19.358621 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:19.358109 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e-metrics-certs\") pod \"network-metrics-daemon-d2sbz\" (UID: \"c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e\") " pod="openshift-multus/network-metrics-daemon-d2sbz" Sep 10 02:49:19.358621 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:19.358233 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 02:49:19.358621 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:19.358248 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 02:49:19.358621 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:19.358314 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e-metrics-certs podName:c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e nodeName:}" failed. No retries permitted until 2026-09-10 02:49:35.358293337 +0000 UTC m=+34.205122604 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e-metrics-certs") pod "network-metrics-daemon-d2sbz" (UID: "c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 02:49:19.358621 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:19.358332 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ff1e29e-7033-490c-9316-def0076b8f7e-original-pull-secret podName:9ff1e29e-7033-490c-9316-def0076b8f7e nodeName:}" failed. No retries permitted until 2026-09-10 02:49:35.358322756 +0000 UTC m=+34.205152008 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9ff1e29e-7033-490c-9316-def0076b8f7e-original-pull-secret") pod "global-pull-secret-syncer-sdxng" (UID: "9ff1e29e-7033-490c-9316-def0076b8f7e") : object "kube-system"/"original-pull-secret" not registered Sep 10 02:49:19.458714 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:19.458663 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt95b\" (UniqueName: \"kubernetes.io/projected/690238d6-9e65-42b8-b042-d1c3efbc1e45-kube-api-access-pt95b\") pod \"network-check-target-fqfck\" (UID: \"690238d6-9e65-42b8-b042-d1c3efbc1e45\") " pod="openshift-network-diagnostics/network-check-target-fqfck" Sep 10 02:49:19.458913 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:19.458862 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 02:49:19.458913 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:19.458891 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 02:49:19.458913 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:19.458905 2560 projected.go:196] Error preparing data for projected volume kube-api-access-pt95b for pod openshift-network-diagnostics/network-check-target-fqfck: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 02:49:19.459057 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:19.458976 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/690238d6-9e65-42b8-b042-d1c3efbc1e45-kube-api-access-pt95b podName:690238d6-9e65-42b8-b042-d1c3efbc1e45 nodeName:}" failed. No retries permitted until 2026-09-10 02:49:35.458956149 +0000 UTC m=+34.305785417 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-pt95b" (UniqueName: "kubernetes.io/projected/690238d6-9e65-42b8-b042-d1c3efbc1e45-kube-api-access-pt95b") pod "network-check-target-fqfck" (UID: "690238d6-9e65-42b8-b042-d1c3efbc1e45") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 02:49:19.640305 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:19.640221 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2sbz" Sep 10 02:49:19.640305 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:19.640266 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fqfck" Sep 10 02:49:19.640548 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:19.640231 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sdxng" Sep 10 02:49:19.640548 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:19.640358 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d2sbz" podUID="c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e" Sep 10 02:49:19.640548 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:19.640474 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fqfck" podUID="690238d6-9e65-42b8-b042-d1c3efbc1e45" Sep 10 02:49:19.640685 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:19.640574 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sdxng" podUID="9ff1e29e-7033-490c-9316-def0076b8f7e" Sep 10 02:49:21.640635 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:21.640599 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2sbz" Sep 10 02:49:21.641083 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:21.640708 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sdxng" Sep 10 02:49:21.641083 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:21.640770 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d2sbz" podUID="c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e" Sep 10 02:49:21.641083 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:21.640815 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sdxng" podUID="9ff1e29e-7033-490c-9316-def0076b8f7e" Sep 10 02:49:21.641083 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:21.640870 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fqfck" Sep 10 02:49:21.641083 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:21.640933 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fqfck" podUID="690238d6-9e65-42b8-b042-d1c3efbc1e45" Sep 10 02:49:23.640446 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:23.640159 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2sbz" Sep 10 02:49:23.640830 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:23.640219 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fqfck" Sep 10 02:49:23.640830 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:23.640573 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d2sbz" podUID="c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e" Sep 10 02:49:23.640830 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:23.640674 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fqfck" podUID="690238d6-9e65-42b8-b042-d1c3efbc1e45" Sep 10 02:49:23.640830 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:23.640250 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sdxng" Sep 10 02:49:23.640830 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:23.640805 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sdxng" podUID="9ff1e29e-7033-490c-9316-def0076b8f7e" Sep 10 02:49:23.741658 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:23.741628 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-z9d9b" event={"ID":"27be8319-a821-4e27-9dc8-411257676cb7","Type":"ContainerStarted","Data":"9f17f1e9af7c134ae7b8cdbb262474db49972eb1da2e6fb2e052058e8782f8ee"} Sep 10 02:49:23.743485 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:23.743460 2560 generic.go:356] "Generic (PLEG): container finished" podID="d2635452-ab19-4377-93d4-3db8d6e0aa88" containerID="b89143180738c76bda6210060545f9352757ed6da780334035092eddeae2307a" exitCode=0 Sep 10 02:49:23.743589 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:23.743536 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6d6zf" event={"ID":"d2635452-ab19-4377-93d4-3db8d6e0aa88","Type":"ContainerDied","Data":"b89143180738c76bda6210060545f9352757ed6da780334035092eddeae2307a"} Sep 10 02:49:23.745209 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:23.745177 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p6kp7" event={"ID":"76f77578-08ad-40d2-849d-7d71c6b7d76b","Type":"ContainerStarted","Data":"4619f790b14ce1b7b3d5c3aa723f47531078714a38aa0f8e6a49d1257e82404a"} Sep 10 02:49:23.746886 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:23.746864 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kkzdh" event={"ID":"6fdc2579-93fb-494d-b179-46cf1db9aaa8","Type":"ContainerStarted","Data":"c02c7d942ac29053f624bf24ab0962b98fc0e096e1ff84245ed0cb772010d336"} Sep 10 02:49:23.749557 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:23.749539 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swhbm_8a6680c4-21b8-4fe2-a763-e84858c7ef90/ovn-acl-logging/0.log" Sep 10 02:49:23.749935 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:23.749914 2560 generic.go:356] "Generic (PLEG): container finished" podID="8a6680c4-21b8-4fe2-a763-e84858c7ef90" containerID="afeb47644b68df655781fa9cdcd9babf5572fcc6f319a8d07a65eac0a86a5eab" exitCode=1 Sep 10 02:49:23.750022 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:23.749982 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" event={"ID":"8a6680c4-21b8-4fe2-a763-e84858c7ef90","Type":"ContainerStarted","Data":"ed07f417697f16bcaf5d4a0c178821f14aca9f2be66e20980e2b509b4f1487f9"} Sep 10 02:49:23.750074 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:23.750053 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" event={"ID":"8a6680c4-21b8-4fe2-a763-e84858c7ef90","Type":"ContainerStarted","Data":"82e030c300611e9cb41ee9156560123efda450625bfe8365add798cff4191f83"} Sep 10 02:49:23.750106 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:23.750086 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" event={"ID":"8a6680c4-21b8-4fe2-a763-e84858c7ef90","Type":"ContainerDied","Data":"afeb47644b68df655781fa9cdcd9babf5572fcc6f319a8d07a65eac0a86a5eab"} Sep 10 02:49:23.750138 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:23.750117 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" event={"ID":"8a6680c4-21b8-4fe2-a763-e84858c7ef90","Type":"ContainerStarted","Data":"1daeb8524d32aef79e09b109a05cddeff56015619cd03c208903cb4f7b9548bf"} Sep 10 02:49:23.751862 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:23.751833 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mnpps" event={"ID":"936410ac-13a7-43ca-a51f-b0bd96649ff6","Type":"ContainerStarted","Data":"7e30805188311ad6af2cd99dcc7e73ab140818059c3c26ab183e0c5dcd65688e"} Sep 10 02:49:23.753320 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:23.753301 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-28crk" event={"ID":"8c69b77f-74ff-4667-9c26-9beba2e52e21","Type":"ContainerStarted","Data":"b16cc5520d20c3dc847a029ad9b3b061682aea2a7b1eabb77462ee6ffec29440"} Sep 10 02:49:23.754737 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:23.754719 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xbp79" event={"ID":"c0f2ef31-35db-4d61-b0c4-77358e82154f","Type":"ContainerStarted","Data":"f2338ca4ac489e0301ca17087ba726e0dab0f80ed41aa5d382e6ab0ec7d28b9e"} Sep 10 02:49:23.760400 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:23.760365 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-37.ec2.internal" podStartSLOduration=20.760337249 podStartE2EDuration="20.760337249s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 02:49:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:49:07.731269824 +0000 UTC m=+6.578099095" watchObservedRunningTime="2026-09-10 02:49:23.760337249 +0000 UTC m=+22.607166510" Sep 10 02:49:23.761002 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:23.760965 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-z9d9b" podStartSLOduration=2.5078078489999998 podStartE2EDuration="21.760955002s" totalImagesPullingTime="19.253147153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 02:49:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:49:23.760311578 +0000 UTC m=+22.607140847" watchObservedRunningTime="2026-09-10 02:49:23.760955002 +0000 UTC m=+22.607784269" Sep 10 02:49:23.799369 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:23.799327 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-kkzdh" podStartSLOduration=2.552362626 podStartE2EDuration="21.799312799s" totalImagesPullingTime="19.246950173s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 02:49:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:49:23.799065304 +0000 UTC m=+22.645894573" watchObservedRunningTime="2026-09-10 02:49:23.799312799 +0000 UTC m=+22.646142061" Sep 10 02:49:23.820598 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:23.820059 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xbp79" podStartSLOduration=2.5899415550000002 podStartE2EDuration="21.820040607s" totalImagesPullingTime="19.230099052s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 02:49:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:49:23.819559003 +0000 UTC m=+22.666388272" watchObservedRunningTime="2026-09-10 02:49:23.820040607 +0000 UTC m=+22.666869876" Sep 10 02:49:23.838646 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:23.838601 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-28crk" podStartSLOduration=2.579938309 podStartE2EDuration="21.83858607s" totalImagesPullingTime="19.258647761s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 02:49:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:49:23.838246338 +0000 UTC m=+22.685075617" watchObservedRunningTime="2026-09-10 02:49:23.83858607 +0000 UTC m=+22.685415337" Sep 10 02:49:23.860101 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:23.860049 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-mnpps" podStartSLOduration=10.224101713 podStartE2EDuration="21.860036979s" totalImagesPullingTime="11.635935266s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 02:49:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:49:23.859805056 +0000 UTC m=+22.706634335" watchObservedRunningTime="2026-09-10 02:49:23.860036979 +0000 UTC m=+22.706866288" Sep 10 02:49:24.760580 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:24.760535 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swhbm_8a6680c4-21b8-4fe2-a763-e84858c7ef90/ovn-acl-logging/0.log" Sep 10 02:49:24.761753 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:24.761020 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" event={"ID":"8a6680c4-21b8-4fe2-a763-e84858c7ef90","Type":"ContainerStarted","Data":"27bd6c3904efe1bb109368128907bb7cfe4834fbb261bb0a295cd26f1ed7d384"} Sep 10 02:49:24.761753 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:24.761139 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" event={"ID":"8a6680c4-21b8-4fe2-a763-e84858c7ef90","Type":"ContainerStarted","Data":"6243407bcbd0776c7128606a15a3dc5610ffdec87759519bacf201b850600128"} Sep 10 02:49:24.762874 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:24.762826 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fwtq4" event={"ID":"078d27a8-8e62-4092-943b-ade84d558613","Type":"ContainerStarted","Data":"eb6290c5450b37675abce83060fd7204d85d8c94f09579365b7dcfbc0879054a"} Sep 10 02:49:24.779955 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:24.779906 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-fwtq4" podStartSLOduration=3.484215498 podStartE2EDuration="22.7798883s" totalImagesPullingTime="19.295672802s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 02:49:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:49:24.779556389 +0000 UTC m=+23.626385661" watchObservedRunningTime="2026-09-10 02:49:24.7798883 +0000 UTC m=+23.626717570" Sep 10 02:49:24.922235 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:24.922206 2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 10 02:49:25.568734 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:25.568604 2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T02:49:24.922231675Z","UUID":"40b475b5-3f93-4c46-bbe5-edbdd1f47053","Handler":null,"Name":"","Endpoint":""} Sep 10 02:49:25.570342 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:25.570310 2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 10 02:49:25.570527 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:25.570348 2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 10 02:49:25.640318 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:25.640287 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sdxng" Sep 10 02:49:25.640540 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:25.640286 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2sbz" Sep 10 02:49:25.640540 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:25.640420 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sdxng" podUID="9ff1e29e-7033-490c-9316-def0076b8f7e" Sep 10 02:49:25.640540 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:25.640292 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fqfck" Sep 10 02:49:25.640540 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:25.640533 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d2sbz" podUID="c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e" Sep 10 02:49:25.640762 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:25.640596 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fqfck" podUID="690238d6-9e65-42b8-b042-d1c3efbc1e45" Sep 10 02:49:25.768013 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:25.767972 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p6kp7" event={"ID":"76f77578-08ad-40d2-849d-7d71c6b7d76b","Type":"ContainerStarted","Data":"c2f46a1a8c7c5563198913a3ff951ec94f1533c50110fd1f782728051801ba59"} Sep 10 02:49:26.774006 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:26.773971 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swhbm_8a6680c4-21b8-4fe2-a763-e84858c7ef90/ovn-acl-logging/0.log" Sep 10 02:49:26.774474 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:26.774447 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" event={"ID":"8a6680c4-21b8-4fe2-a763-e84858c7ef90","Type":"ContainerStarted","Data":"bb6b516b2f823a1cd7e383e817f1c797f847718dccd49dfd4897da990a910fe1"} Sep 10 02:49:26.776795 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:26.776767 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p6kp7" event={"ID":"76f77578-08ad-40d2-849d-7d71c6b7d76b","Type":"ContainerStarted","Data":"73599ad5cab7fc9f23a8a1aeaac18aa19994e6a819986d0ba7c1583a7d972d9b"} Sep 10 02:49:26.798728 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:26.798677 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p6kp7" podStartSLOduration=3.251522697 podStartE2EDuration="24.798662051s" totalImagesPullingTime="21.547139354s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 02:49:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:49:26.798228239 +0000 UTC m=+25.645057506" watchObservedRunningTime="2026-09-10 02:49:26.798662051 +0000 UTC m=+25.645491318" Sep 10 02:49:27.563948 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:27.563702 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-z9d9b" Sep 10 02:49:27.564631 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:27.564606 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-z9d9b" Sep 10 02:49:27.640838 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:27.640804 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2sbz" Sep 10 02:49:27.640838 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:27.640842 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fqfck" Sep 10 02:49:27.641036 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:27.640817 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sdxng" Sep 10 02:49:27.641036 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:27.640936 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d2sbz" podUID="c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e" Sep 10 02:49:27.641036 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:27.641007 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fqfck" podUID="690238d6-9e65-42b8-b042-d1c3efbc1e45" Sep 10 02:49:27.641147 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:27.641082 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sdxng" podUID="9ff1e29e-7033-490c-9316-def0076b8f7e" Sep 10 02:49:27.779660 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:27.779614 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-z9d9b" Sep 10 02:49:27.780075 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:27.779967 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-z9d9b" Sep 10 02:49:28.785894 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:28.785866 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swhbm_8a6680c4-21b8-4fe2-a763-e84858c7ef90/ovn-acl-logging/0.log" Sep 10 02:49:28.786342 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:28.786314 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" event={"ID":"8a6680c4-21b8-4fe2-a763-e84858c7ef90","Type":"ContainerStarted","Data":"138f88e8184845057fb1be98628cfbfa8f43f5bbe0bdb497898c05fc2d579027"} Sep 10 02:49:28.786773 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:28.786754 2560 scope.go:117] "RemoveContainer" containerID="afeb47644b68df655781fa9cdcd9babf5572fcc6f319a8d07a65eac0a86a5eab" Sep 10 02:49:29.640740 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:29.640426 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2sbz" Sep 10 02:49:29.640740 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:29.640426 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sdxng" Sep 10 02:49:29.640860 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:29.640747 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d2sbz" podUID="c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e" Sep 10 02:49:29.640860 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:29.640459 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fqfck" Sep 10 02:49:29.640860 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:29.640787 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sdxng" podUID="9ff1e29e-7033-490c-9316-def0076b8f7e" Sep 10 02:49:29.640994 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:29.640861 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fqfck" podUID="690238d6-9e65-42b8-b042-d1c3efbc1e45" Sep 10 02:49:29.792550 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:29.792525 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swhbm_8a6680c4-21b8-4fe2-a763-e84858c7ef90/ovn-acl-logging/0.log" Sep 10 02:49:29.792963 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:29.792937 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" event={"ID":"8a6680c4-21b8-4fe2-a763-e84858c7ef90","Type":"ContainerStarted","Data":"d69eae73376a31585180ce088cfe2d8f7ae6f8538ae03c877388ed53319dff88"} Sep 10 02:49:29.793178 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:29.793157 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:29.793293 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:29.793212 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:29.793293 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:29.793259 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:29.795081 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:29.795056 2560 generic.go:356] "Generic (PLEG): container finished" podID="d2635452-ab19-4377-93d4-3db8d6e0aa88" containerID="a20dc55abfec022223a568941a31f704ff94ecdc68ae8ce0a205baf7ce7e8ee0" exitCode=0 Sep 10 02:49:29.795183 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:29.795089 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6d6zf" event={"ID":"d2635452-ab19-4377-93d4-3db8d6e0aa88","Type":"ContainerDied","Data":"a20dc55abfec022223a568941a31f704ff94ecdc68ae8ce0a205baf7ce7e8ee0"} Sep 10 02:49:29.808977 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:29.808952 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:29.809091 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:29.809082 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:49:29.829868 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:29.829819 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" podStartSLOduration=8.541206265 podStartE2EDuration="27.829803704s" totalImagesPullingTime="19.288597439s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 02:49:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:49:29.82942761 +0000 UTC m=+28.676256877" watchObservedRunningTime="2026-09-10 02:49:29.829803704 +0000 UTC m=+28.676632972" Sep 10 02:49:30.580097 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:30.580064 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sdxng"] Sep 10 02:49:30.580243 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:30.580224 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sdxng" Sep 10 02:49:30.580361 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:30.580337 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sdxng" podUID="9ff1e29e-7033-490c-9316-def0076b8f7e" Sep 10 02:49:30.583319 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:30.582813 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-d2sbz"] Sep 10 02:49:30.583319 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:30.582955 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2sbz" Sep 10 02:49:30.583319 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:30.583052 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d2sbz" podUID="c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e" Sep 10 02:49:30.583632 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:30.583610 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fqfck"] Sep 10 02:49:30.583718 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:30.583707 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fqfck" Sep 10 02:49:30.583818 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:30.583796 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fqfck" podUID="690238d6-9e65-42b8-b042-d1c3efbc1e45" Sep 10 02:49:30.799394 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:30.799363 2560 generic.go:356] "Generic (PLEG): container finished" podID="d2635452-ab19-4377-93d4-3db8d6e0aa88" containerID="ed576b2660910c0ddd43b83d05f8596792923497c49cbcbaeb716e5560effd8f" exitCode=0 Sep 10 02:49:30.800194 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:30.799440 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6d6zf" event={"ID":"d2635452-ab19-4377-93d4-3db8d6e0aa88","Type":"ContainerDied","Data":"ed576b2660910c0ddd43b83d05f8596792923497c49cbcbaeb716e5560effd8f"} Sep 10 02:49:31.804198 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:31.804160 2560 generic.go:356] "Generic (PLEG): container finished" podID="d2635452-ab19-4377-93d4-3db8d6e0aa88" containerID="639d773016fc83a50a30ea565eba32ce28ba2b98ab85f3247ad11d7979e6f6cb" exitCode=0 Sep 10 02:49:31.804599 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:31.804245 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6d6zf" event={"ID":"d2635452-ab19-4377-93d4-3db8d6e0aa88","Type":"ContainerDied","Data":"639d773016fc83a50a30ea565eba32ce28ba2b98ab85f3247ad11d7979e6f6cb"} Sep 10 02:49:32.640551 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:32.640498 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fqfck" Sep 10 02:49:32.640551 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:32.640545 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2sbz" Sep 10 02:49:32.640758 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:32.640640 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fqfck" podUID="690238d6-9e65-42b8-b042-d1c3efbc1e45" Sep 10 02:49:32.640758 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:32.640700 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sdxng" Sep 10 02:49:32.640865 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:32.640831 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d2sbz" podUID="c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e" Sep 10 02:49:32.640967 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:32.640905 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sdxng" podUID="9ff1e29e-7033-490c-9316-def0076b8f7e" Sep 10 02:49:34.640893 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:34.640571 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fqfck" Sep 10 02:49:34.640893 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:34.640603 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2sbz" Sep 10 02:49:34.640893 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:34.640603 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sdxng" Sep 10 02:49:34.641482 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:34.640938 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fqfck" podUID="690238d6-9e65-42b8-b042-d1c3efbc1e45" Sep 10 02:49:34.641482 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:34.641021 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d2sbz" podUID="c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e" Sep 10 02:49:34.641482 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:34.641115 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sdxng" podUID="9ff1e29e-7033-490c-9316-def0076b8f7e" Sep 10 02:49:35.370577 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:35.370503 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ff1e29e-7033-490c-9316-def0076b8f7e-original-pull-secret\") pod \"global-pull-secret-syncer-sdxng\" (UID: \"9ff1e29e-7033-490c-9316-def0076b8f7e\") " pod="kube-system/global-pull-secret-syncer-sdxng" Sep 10 02:49:35.370877 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:35.370597 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e-metrics-certs\") pod \"network-metrics-daemon-d2sbz\" (UID: \"c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e\") " pod="openshift-multus/network-metrics-daemon-d2sbz" Sep 10 02:49:35.370877 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:35.370636 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 02:49:35.370877 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:35.370720 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 02:49:35.370877 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:35.370724 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ff1e29e-7033-490c-9316-def0076b8f7e-original-pull-secret podName:9ff1e29e-7033-490c-9316-def0076b8f7e nodeName:}" failed. No retries permitted until 2026-09-10 02:50:07.37070293 +0000 UTC m=+66.217532199 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9ff1e29e-7033-490c-9316-def0076b8f7e-original-pull-secret") pod "global-pull-secret-syncer-sdxng" (UID: "9ff1e29e-7033-490c-9316-def0076b8f7e") : object "kube-system"/"original-pull-secret" not registered Sep 10 02:49:35.370877 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:35.370765 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e-metrics-certs podName:c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e nodeName:}" failed. No retries permitted until 2026-09-10 02:50:07.370750208 +0000 UTC m=+66.217579468 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e-metrics-certs") pod "network-metrics-daemon-d2sbz" (UID: "c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 02:49:35.471524 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:35.471476 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt95b\" (UniqueName: \"kubernetes.io/projected/690238d6-9e65-42b8-b042-d1c3efbc1e45-kube-api-access-pt95b\") pod \"network-check-target-fqfck\" (UID: \"690238d6-9e65-42b8-b042-d1c3efbc1e45\") " pod="openshift-network-diagnostics/network-check-target-fqfck" Sep 10 02:49:35.471704 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:35.471613 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 02:49:35.471704 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:35.471636 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 02:49:35.471704 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:35.471646 2560 projected.go:196] Error preparing data for projected volume kube-api-access-pt95b for pod openshift-network-diagnostics/network-check-target-fqfck: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 02:49:35.471841 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:35.471705 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/690238d6-9e65-42b8-b042-d1c3efbc1e45-kube-api-access-pt95b podName:690238d6-9e65-42b8-b042-d1c3efbc1e45 nodeName:}" failed. No retries permitted until 2026-09-10 02:50:07.471690872 +0000 UTC m=+66.318520138 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-pt95b" (UniqueName: "kubernetes.io/projected/690238d6-9e65-42b8-b042-d1c3efbc1e45-kube-api-access-pt95b") pod "network-check-target-fqfck" (UID: "690238d6-9e65-42b8-b042-d1c3efbc1e45") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 02:49:35.986613 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:35.986583 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-37.ec2.internal" event="NodeReady" Sep 10 02:49:35.987023 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:35.986749 2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 10 02:49:36.031827 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.031751 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-75f8b47547-m9np9"] Sep 10 02:49:36.053190 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.053163 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2s6nr"] Sep 10 02:49:36.053396 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.053266 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75f8b47547-m9np9" Sep 10 02:49:36.056089 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.056067 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 10 02:49:36.056212 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.056112 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 10 02:49:36.056633 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.056613 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 10 02:49:36.056735 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.056709 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-glllk\"" Sep 10 02:49:36.066500 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.066479 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 10 02:49:36.076584 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.076565 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-75f8b47547-m9np9"] Sep 10 02:49:36.076679 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.076596 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-mpnm6"] Sep 10 02:49:36.076718 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.076696 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2s6nr" Sep 10 02:49:36.084415 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.082615 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-nh6lc\"" Sep 10 02:49:36.084415 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.082734 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 10 02:49:36.084415 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.082989 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 10 02:49:36.092296 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.092277 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2s6nr"] Sep 10 02:49:36.092389 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.092308 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mpnm6"] Sep 10 02:49:36.092389 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.092379 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mpnm6" Sep 10 02:49:36.095141 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.095123 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 10 02:49:36.095232 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.095144 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dfrfr\"" Sep 10 02:49:36.095232 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.095201 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 10 02:49:36.095434 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.095420 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 10 02:49:36.178134 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.178094 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-bound-sa-token\") pod \"image-registry-75f8b47547-m9np9\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " pod="openshift-image-registry/image-registry-75f8b47547-m9np9" Sep 10 02:49:36.178134 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.178137 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e4d56e77-0fdc-4733-b06f-2451d59f8343-image-registry-private-configuration\") pod \"image-registry-75f8b47547-m9np9\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " pod="openshift-image-registry/image-registry-75f8b47547-m9np9" Sep 10 02:49:36.178344 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.178164 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-registry-tls\") pod \"image-registry-75f8b47547-m9np9\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " pod="openshift-image-registry/image-registry-75f8b47547-m9np9" Sep 10 02:49:36.178344 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.178187 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5smb\" (UniqueName: \"kubernetes.io/projected/98643336-46ce-4f17-acb2-d8110a5bfb84-kube-api-access-z5smb\") pod \"dns-default-2s6nr\" (UID: \"98643336-46ce-4f17-acb2-d8110a5bfb84\") " pod="openshift-dns/dns-default-2s6nr" Sep 10 02:49:36.178344 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.178222 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e4d56e77-0fdc-4733-b06f-2451d59f8343-ca-trust-extracted\") pod \"image-registry-75f8b47547-m9np9\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " pod="openshift-image-registry/image-registry-75f8b47547-m9np9" Sep 10 02:49:36.178344 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.178239 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e4d56e77-0fdc-4733-b06f-2451d59f8343-installation-pull-secrets\") pod \"image-registry-75f8b47547-m9np9\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " pod="openshift-image-registry/image-registry-75f8b47547-m9np9" Sep 10 02:49:36.178344 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.178272 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98643336-46ce-4f17-acb2-d8110a5bfb84-tmp-dir\") pod \"dns-default-2s6nr\" (UID: \"98643336-46ce-4f17-acb2-d8110a5bfb84\") " pod="openshift-dns/dns-default-2s6nr" Sep 10 02:49:36.178344 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.178299 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/007e1c44-e993-4808-b999-05799e398c2f-cert\") pod \"ingress-canary-mpnm6\" (UID: \"007e1c44-e993-4808-b999-05799e398c2f\") " pod="openshift-ingress-canary/ingress-canary-mpnm6" Sep 10 02:49:36.178344 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.178314 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e4d56e77-0fdc-4733-b06f-2451d59f8343-trusted-ca\") pod \"image-registry-75f8b47547-m9np9\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " pod="openshift-image-registry/image-registry-75f8b47547-m9np9" Sep 10 02:49:36.178614 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.178354 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/98643336-46ce-4f17-acb2-d8110a5bfb84-config-volume\") pod \"dns-default-2s6nr\" (UID: \"98643336-46ce-4f17-acb2-d8110a5bfb84\") " pod="openshift-dns/dns-default-2s6nr" Sep 10 02:49:36.178614 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.178386 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e4d56e77-0fdc-4733-b06f-2451d59f8343-registry-certificates\") pod \"image-registry-75f8b47547-m9np9\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " pod="openshift-image-registry/image-registry-75f8b47547-m9np9" Sep 10 02:49:36.178614 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.178402 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-854fs\" (UniqueName: \"kubernetes.io/projected/007e1c44-e993-4808-b999-05799e398c2f-kube-api-access-854fs\") pod \"ingress-canary-mpnm6\" (UID: \"007e1c44-e993-4808-b999-05799e398c2f\") " pod="openshift-ingress-canary/ingress-canary-mpnm6" Sep 10 02:49:36.178614 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.178420 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnc2k\" (UniqueName: \"kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-kube-api-access-gnc2k\") pod \"image-registry-75f8b47547-m9np9\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " pod="openshift-image-registry/image-registry-75f8b47547-m9np9" Sep 10 02:49:36.178614 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.178435 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/98643336-46ce-4f17-acb2-d8110a5bfb84-metrics-tls\") pod \"dns-default-2s6nr\" (UID: \"98643336-46ce-4f17-acb2-d8110a5bfb84\") " pod="openshift-dns/dns-default-2s6nr" Sep 10 02:49:36.246222 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.246187 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-z9d9b_27be8319-a821-4e27-9dc8-411257676cb7/konnectivity-agent/0.log" limit=1024 Sep 10 02:49:36.279552 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.279521 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/98643336-46ce-4f17-acb2-d8110a5bfb84-config-volume\") pod \"dns-default-2s6nr\" (UID: \"98643336-46ce-4f17-acb2-d8110a5bfb84\") " pod="openshift-dns/dns-default-2s6nr" Sep 10 02:49:36.279665 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.279588 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e4d56e77-0fdc-4733-b06f-2451d59f8343-registry-certificates\") pod \"image-registry-75f8b47547-m9np9\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " pod="openshift-image-registry/image-registry-75f8b47547-m9np9" Sep 10 02:49:36.279665 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.279623 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-854fs\" (UniqueName: \"kubernetes.io/projected/007e1c44-e993-4808-b999-05799e398c2f-kube-api-access-854fs\") pod \"ingress-canary-mpnm6\" (UID: \"007e1c44-e993-4808-b999-05799e398c2f\") " pod="openshift-ingress-canary/ingress-canary-mpnm6" Sep 10 02:49:36.279665 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.279655 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnc2k\" (UniqueName: \"kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-kube-api-access-gnc2k\") pod \"image-registry-75f8b47547-m9np9\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " pod="openshift-image-registry/image-registry-75f8b47547-m9np9" Sep 10 02:49:36.279788 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.279681 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/98643336-46ce-4f17-acb2-d8110a5bfb84-metrics-tls\") pod \"dns-default-2s6nr\" (UID: \"98643336-46ce-4f17-acb2-d8110a5bfb84\") " pod="openshift-dns/dns-default-2s6nr" Sep 10 02:49:36.279788 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.279736 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-bound-sa-token\") pod \"image-registry-75f8b47547-m9np9\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " pod="openshift-image-registry/image-registry-75f8b47547-m9np9" Sep 10 02:49:36.279788 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.279765 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e4d56e77-0fdc-4733-b06f-2451d59f8343-image-registry-private-configuration\") pod \"image-registry-75f8b47547-m9np9\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " pod="openshift-image-registry/image-registry-75f8b47547-m9np9" Sep 10 02:49:36.279925 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.279791 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-registry-tls\") pod \"image-registry-75f8b47547-m9np9\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " pod="openshift-image-registry/image-registry-75f8b47547-m9np9" Sep 10 02:49:36.279925 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.279815 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5smb\" (UniqueName: \"kubernetes.io/projected/98643336-46ce-4f17-acb2-d8110a5bfb84-kube-api-access-z5smb\") pod \"dns-default-2s6nr\" (UID: \"98643336-46ce-4f17-acb2-d8110a5bfb84\") " pod="openshift-dns/dns-default-2s6nr" Sep 10 02:49:36.279925 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.279841 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e4d56e77-0fdc-4733-b06f-2451d59f8343-ca-trust-extracted\") pod \"image-registry-75f8b47547-m9np9\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " pod="openshift-image-registry/image-registry-75f8b47547-m9np9" Sep 10 02:49:36.279925 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.279866 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e4d56e77-0fdc-4733-b06f-2451d59f8343-installation-pull-secrets\") pod \"image-registry-75f8b47547-m9np9\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " pod="openshift-image-registry/image-registry-75f8b47547-m9np9" Sep 10 02:49:36.279925 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.279908 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98643336-46ce-4f17-acb2-d8110a5bfb84-tmp-dir\") pod \"dns-default-2s6nr\" (UID: \"98643336-46ce-4f17-acb2-d8110a5bfb84\") " pod="openshift-dns/dns-default-2s6nr" Sep 10 02:49:36.280165 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.279948 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/007e1c44-e993-4808-b999-05799e398c2f-cert\") pod \"ingress-canary-mpnm6\" (UID: \"007e1c44-e993-4808-b999-05799e398c2f\") " pod="openshift-ingress-canary/ingress-canary-mpnm6" Sep 10 02:49:36.280165 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.279969 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e4d56e77-0fdc-4733-b06f-2451d59f8343-trusted-ca\") pod \"image-registry-75f8b47547-m9np9\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " pod="openshift-image-registry/image-registry-75f8b47547-m9np9" Sep 10 02:49:36.280165 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.280127 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/98643336-46ce-4f17-acb2-d8110a5bfb84-config-volume\") pod \"dns-default-2s6nr\" (UID: \"98643336-46ce-4f17-acb2-d8110a5bfb84\") " pod="openshift-dns/dns-default-2s6nr" Sep 10 02:49:36.280311 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.280234 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e4d56e77-0fdc-4733-b06f-2451d59f8343-registry-certificates\") pod \"image-registry-75f8b47547-m9np9\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " pod="openshift-image-registry/image-registry-75f8b47547-m9np9" Sep 10 02:49:36.280311 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:36.279868 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 02:49:36.280311 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.280303 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98643336-46ce-4f17-acb2-d8110a5bfb84-tmp-dir\") pod \"dns-default-2s6nr\" (UID: \"98643336-46ce-4f17-acb2-d8110a5bfb84\") " pod="openshift-dns/dns-default-2s6nr" Sep 10 02:49:36.280445 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:36.280311 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98643336-46ce-4f17-acb2-d8110a5bfb84-metrics-tls podName:98643336-46ce-4f17-acb2-d8110a5bfb84 nodeName:}" failed. No retries permitted until 2026-09-10 02:49:36.780291477 +0000 UTC m=+35.627120736 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/98643336-46ce-4f17-acb2-d8110a5bfb84-metrics-tls") pod "dns-default-2s6nr" (UID: "98643336-46ce-4f17-acb2-d8110a5bfb84") : secret "dns-default-metrics-tls" not found Sep 10 02:49:36.280445 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.280396 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e4d56e77-0fdc-4733-b06f-2451d59f8343-ca-trust-extracted\") pod \"image-registry-75f8b47547-m9np9\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " pod="openshift-image-registry/image-registry-75f8b47547-m9np9" Sep 10 02:49:36.280588 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:36.280468 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 02:49:36.280588 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:36.280481 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75f8b47547-m9np9: secret "image-registry-tls" not found Sep 10 02:49:36.280588 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:36.280555 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-registry-tls podName:e4d56e77-0fdc-4733-b06f-2451d59f8343 nodeName:}" failed. No retries permitted until 2026-09-10 02:49:36.780538099 +0000 UTC m=+35.627367359 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-registry-tls") pod "image-registry-75f8b47547-m9np9" (UID: "e4d56e77-0fdc-4733-b06f-2451d59f8343") : secret "image-registry-tls" not found Sep 10 02:49:36.280883 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.280864 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e4d56e77-0fdc-4733-b06f-2451d59f8343-trusted-ca\") pod \"image-registry-75f8b47547-m9np9\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " pod="openshift-image-registry/image-registry-75f8b47547-m9np9" Sep 10 02:49:36.284083 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.284061 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e4d56e77-0fdc-4733-b06f-2451d59f8343-installation-pull-secrets\") pod \"image-registry-75f8b47547-m9np9\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " pod="openshift-image-registry/image-registry-75f8b47547-m9np9" Sep 10 02:49:36.284162 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.284079 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e4d56e77-0fdc-4733-b06f-2451d59f8343-image-registry-private-configuration\") pod \"image-registry-75f8b47547-m9np9\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " pod="openshift-image-registry/image-registry-75f8b47547-m9np9" Sep 10 02:49:36.284373 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.284350 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/007e1c44-e993-4808-b999-05799e398c2f-cert\") pod \"ingress-canary-mpnm6\" (UID: \"007e1c44-e993-4808-b999-05799e398c2f\") " pod="openshift-ingress-canary/ingress-canary-mpnm6" Sep 10 02:49:36.293169 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.293145 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnc2k\" (UniqueName: \"kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-kube-api-access-gnc2k\") pod \"image-registry-75f8b47547-m9np9\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " pod="openshift-image-registry/image-registry-75f8b47547-m9np9" Sep 10 02:49:36.295636 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.295483 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-bound-sa-token\") pod \"image-registry-75f8b47547-m9np9\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " pod="openshift-image-registry/image-registry-75f8b47547-m9np9" Sep 10 02:49:36.297173 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.297039 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-854fs\" (UniqueName: \"kubernetes.io/projected/007e1c44-e993-4808-b999-05799e398c2f-kube-api-access-854fs\") pod \"ingress-canary-mpnm6\" (UID: \"007e1c44-e993-4808-b999-05799e398c2f\") " pod="openshift-ingress-canary/ingress-canary-mpnm6" Sep 10 02:49:36.297269 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.297194 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5smb\" (UniqueName: \"kubernetes.io/projected/98643336-46ce-4f17-acb2-d8110a5bfb84-kube-api-access-z5smb\") pod \"dns-default-2s6nr\" (UID: \"98643336-46ce-4f17-acb2-d8110a5bfb84\") " pod="openshift-dns/dns-default-2s6nr" Sep 10 02:49:36.352724 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.352687 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-z9d9b_27be8319-a821-4e27-9dc8-411257676cb7/konnectivity-agent/0.log" limit=1024 Sep 10 02:49:36.400967 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.400737 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mpnm6" Sep 10 02:49:36.640389 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.640342 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2sbz" Sep 10 02:49:36.640389 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.640385 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fqfck" Sep 10 02:49:36.640657 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.640385 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sdxng" Sep 10 02:49:36.644009 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.643989 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 10 02:49:36.644271 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.644251 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 10 02:49:36.644271 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.644263 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 10 02:49:36.644492 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.644477 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lc7sq\"" Sep 10 02:49:36.644569 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.644526 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 02:49:36.644608 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.644586 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j5skv\"" Sep 10 02:49:36.680745 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.680719 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mpnm6"] Sep 10 02:49:36.684978 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:36.684954 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod007e1c44_e993_4808_b999_05799e398c2f.slice/crio-4213405d23b1703afd3c607d58d782d34674849623f2b78376e2d6b1856ca039 WatchSource:0}: Error finding container 4213405d23b1703afd3c607d58d782d34674849623f2b78376e2d6b1856ca039: Status 404 returned error can't find the container with id 4213405d23b1703afd3c607d58d782d34674849623f2b78376e2d6b1856ca039 Sep 10 02:49:36.783939 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.783904 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/98643336-46ce-4f17-acb2-d8110a5bfb84-metrics-tls\") pod \"dns-default-2s6nr\" (UID: \"98643336-46ce-4f17-acb2-d8110a5bfb84\") " pod="openshift-dns/dns-default-2s6nr" Sep 10 02:49:36.784114 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.783958 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-registry-tls\") pod \"image-registry-75f8b47547-m9np9\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " pod="openshift-image-registry/image-registry-75f8b47547-m9np9" Sep 10 02:49:36.784114 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:36.784058 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 02:49:36.784114 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:36.784079 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 02:49:36.784114 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:36.784091 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75f8b47547-m9np9: secret "image-registry-tls" not found Sep 10 02:49:36.784261 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:36.784126 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98643336-46ce-4f17-acb2-d8110a5bfb84-metrics-tls podName:98643336-46ce-4f17-acb2-d8110a5bfb84 nodeName:}" failed. No retries permitted until 2026-09-10 02:49:37.78410775 +0000 UTC m=+36.630937019 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/98643336-46ce-4f17-acb2-d8110a5bfb84-metrics-tls") pod "dns-default-2s6nr" (UID: "98643336-46ce-4f17-acb2-d8110a5bfb84") : secret "dns-default-metrics-tls" not found Sep 10 02:49:36.784261 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:36.784143 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-registry-tls podName:e4d56e77-0fdc-4733-b06f-2451d59f8343 nodeName:}" failed. No retries permitted until 2026-09-10 02:49:37.784134836 +0000 UTC m=+36.630964083 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-registry-tls") pod "image-registry-75f8b47547-m9np9" (UID: "e4d56e77-0fdc-4733-b06f-2451d59f8343") : secret "image-registry-tls" not found Sep 10 02:49:36.819800 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.819761 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mpnm6" event={"ID":"007e1c44-e993-4808-b999-05799e398c2f","Type":"ContainerStarted","Data":"4213405d23b1703afd3c607d58d782d34674849623f2b78376e2d6b1856ca039"} Sep 10 02:49:36.822632 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.822606 2560 generic.go:356] "Generic (PLEG): container finished" podID="d2635452-ab19-4377-93d4-3db8d6e0aa88" containerID="c86c63742bba98c2d13f8e087b3672a4bf7a824014cf60433f77389c8edff6e5" exitCode=0 Sep 10 02:49:36.822744 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:36.822660 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6d6zf" event={"ID":"d2635452-ab19-4377-93d4-3db8d6e0aa88","Type":"ContainerDied","Data":"c86c63742bba98c2d13f8e087b3672a4bf7a824014cf60433f77389c8edff6e5"} Sep 10 02:49:37.408450 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:37.408253 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-z9d9b_27be8319-a821-4e27-9dc8-411257676cb7/konnectivity-agent/0.log" limit=1024 Sep 10 02:49:37.509016 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:37.508927 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-z9d9b_27be8319-a821-4e27-9dc8-411257676cb7/konnectivity-agent/0.log" limit=1024 Sep 10 02:49:37.622270 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:37.622239 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-z9d9b_27be8319-a821-4e27-9dc8-411257676cb7/konnectivity-agent/0.log" limit=1024 Sep 10 02:49:37.792376 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:37.792289 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/98643336-46ce-4f17-acb2-d8110a5bfb84-metrics-tls\") pod \"dns-default-2s6nr\" (UID: \"98643336-46ce-4f17-acb2-d8110a5bfb84\") " pod="openshift-dns/dns-default-2s6nr" Sep 10 02:49:37.792574 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:37.792389 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-registry-tls\") pod \"image-registry-75f8b47547-m9np9\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " pod="openshift-image-registry/image-registry-75f8b47547-m9np9" Sep 10 02:49:37.792574 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:37.792461 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 02:49:37.792574 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:37.792495 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 02:49:37.792574 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:37.792528 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75f8b47547-m9np9: secret "image-registry-tls" not found Sep 10 02:49:37.792574 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:37.792549 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98643336-46ce-4f17-acb2-d8110a5bfb84-metrics-tls podName:98643336-46ce-4f17-acb2-d8110a5bfb84 nodeName:}" failed. No retries permitted until 2026-09-10 02:49:39.792532075 +0000 UTC m=+38.639361324 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/98643336-46ce-4f17-acb2-d8110a5bfb84-metrics-tls") pod "dns-default-2s6nr" (UID: "98643336-46ce-4f17-acb2-d8110a5bfb84") : secret "dns-default-metrics-tls" not found Sep 10 02:49:37.792860 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:37.792635 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-registry-tls podName:e4d56e77-0fdc-4733-b06f-2451d59f8343 nodeName:}" failed. No retries permitted until 2026-09-10 02:49:39.792560404 +0000 UTC m=+38.639389663 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-registry-tls") pod "image-registry-75f8b47547-m9np9" (UID: "e4d56e77-0fdc-4733-b06f-2451d59f8343") : secret "image-registry-tls" not found Sep 10 02:49:37.828859 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:37.828827 2560 generic.go:356] "Generic (PLEG): container finished" podID="d2635452-ab19-4377-93d4-3db8d6e0aa88" containerID="fd8b1d6b6e54d518c6998d719f41a2e593ae100f3834881610c34f71598a9b0d" exitCode=0 Sep 10 02:49:37.829011 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:37.828876 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6d6zf" event={"ID":"d2635452-ab19-4377-93d4-3db8d6e0aa88","Type":"ContainerDied","Data":"fd8b1d6b6e54d518c6998d719f41a2e593ae100f3834881610c34f71598a9b0d"} Sep 10 02:49:38.835448 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:38.835408 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6d6zf" event={"ID":"d2635452-ab19-4377-93d4-3db8d6e0aa88","Type":"ContainerStarted","Data":"284c28f070b567ee4959eb46b8a92b05785656b8f4534591bf5b338135ed22d6"} Sep 10 02:49:38.879893 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:38.879832 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6d6zf" podStartSLOduration=6.671048698 podStartE2EDuration="36.879810512s" totalImagesPullingTime="30.208761814s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 02:49:02 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:49:38.879652066 +0000 UTC m=+37.726481334" watchObservedRunningTime="2026-09-10 02:49:38.879810512 +0000 UTC m=+37.726639780" Sep 10 02:49:39.809529 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:39.809429 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/98643336-46ce-4f17-acb2-d8110a5bfb84-metrics-tls\") pod \"dns-default-2s6nr\" (UID: \"98643336-46ce-4f17-acb2-d8110a5bfb84\") " pod="openshift-dns/dns-default-2s6nr" Sep 10 02:49:39.809529 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:39.809483 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-registry-tls\") pod \"image-registry-75f8b47547-m9np9\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " pod="openshift-image-registry/image-registry-75f8b47547-m9np9" Sep 10 02:49:39.809713 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:39.809680 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 02:49:39.809766 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:39.809750 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98643336-46ce-4f17-acb2-d8110a5bfb84-metrics-tls podName:98643336-46ce-4f17-acb2-d8110a5bfb84 nodeName:}" failed. No retries permitted until 2026-09-10 02:49:43.809734186 +0000 UTC m=+42.656563447 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/98643336-46ce-4f17-acb2-d8110a5bfb84-metrics-tls") pod "dns-default-2s6nr" (UID: "98643336-46ce-4f17-acb2-d8110a5bfb84") : secret "dns-default-metrics-tls" not found Sep 10 02:49:39.809766 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:39.809753 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 02:49:39.809856 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:39.809774 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75f8b47547-m9np9: secret "image-registry-tls" not found Sep 10 02:49:39.809856 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:39.809797 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-registry-tls podName:e4d56e77-0fdc-4733-b06f-2451d59f8343 nodeName:}" failed. No retries permitted until 2026-09-10 02:49:43.809790894 +0000 UTC m=+42.656620189 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-registry-tls") pod "image-registry-75f8b47547-m9np9" (UID: "e4d56e77-0fdc-4733-b06f-2451d59f8343") : secret "image-registry-tls" not found Sep 10 02:49:39.839778 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:39.839738 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mpnm6" event={"ID":"007e1c44-e993-4808-b999-05799e398c2f","Type":"ContainerStarted","Data":"1b2d8170469e1c233b2d2f65cc9a3c8fbfb2f9244356a16f5110b5b8d66a4346"} Sep 10 02:49:39.857538 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:39.857448 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-mpnm6" podStartSLOduration=1.041114561 podStartE2EDuration="3.857429095s" totalImagesPullingTime="2.816314534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 02:49:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:49:39.857124209 +0000 UTC m=+38.703953489" watchObservedRunningTime="2026-09-10 02:49:39.857429095 +0000 UTC m=+38.704258363" Sep 10 02:49:43.839478 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:43.839434 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/98643336-46ce-4f17-acb2-d8110a5bfb84-metrics-tls\") pod \"dns-default-2s6nr\" (UID: \"98643336-46ce-4f17-acb2-d8110a5bfb84\") " pod="openshift-dns/dns-default-2s6nr" Sep 10 02:49:43.839968 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:43.839494 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-registry-tls\") pod \"image-registry-75f8b47547-m9np9\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " pod="openshift-image-registry/image-registry-75f8b47547-m9np9" Sep 10 02:49:43.839968 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:43.839603 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 02:49:43.839968 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:43.839608 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 02:49:43.839968 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:43.839614 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75f8b47547-m9np9: secret "image-registry-tls" not found Sep 10 02:49:43.839968 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:43.839683 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98643336-46ce-4f17-acb2-d8110a5bfb84-metrics-tls podName:98643336-46ce-4f17-acb2-d8110a5bfb84 nodeName:}" failed. No retries permitted until 2026-09-10 02:49:51.839663966 +0000 UTC m=+50.686493232 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/98643336-46ce-4f17-acb2-d8110a5bfb84-metrics-tls") pod "dns-default-2s6nr" (UID: "98643336-46ce-4f17-acb2-d8110a5bfb84") : secret "dns-default-metrics-tls" not found Sep 10 02:49:43.839968 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:43.839709 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-registry-tls podName:e4d56e77-0fdc-4733-b06f-2451d59f8343 nodeName:}" failed. No retries permitted until 2026-09-10 02:49:51.839694953 +0000 UTC m=+50.686524232 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-registry-tls") pod "image-registry-75f8b47547-m9np9" (UID: "e4d56e77-0fdc-4733-b06f-2451d59f8343") : secret "image-registry-tls" not found Sep 10 02:49:49.254044 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.254007 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-699696b566-qzknj"] Sep 10 02:49:49.293161 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.293133 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-699696b566-qzknj" Sep 10 02:49:49.294880 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.294852 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-699696b566-qzknj"] Sep 10 02:49:49.296052 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.296030 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Sep 10 02:49:49.296314 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.296296 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Sep 10 02:49:49.296610 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.296589 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Sep 10 02:49:49.296706 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.296628 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 10 02:49:49.297115 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.297098 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-w6765\"" Sep 10 02:49:49.297187 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.297100 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Sep 10 02:49:49.297187 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.297154 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 10 02:49:49.335479 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.335450 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-dvdwf"] Sep 10 02:49:49.361285 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.361255 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-l89pb"] Sep 10 02:49:49.361440 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.361392 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-dvdwf" Sep 10 02:49:49.364873 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.364847 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Sep 10 02:49:49.365153 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.365133 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Sep 10 02:49:49.365246 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.365186 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-whkjk\"" Sep 10 02:49:49.365314 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.365238 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Sep 10 02:49:49.365892 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.365873 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Sep 10 02:49:49.380348 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.380325 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2szl\" (UniqueName: \"kubernetes.io/projected/9733272a-e52f-47ca-abc7-fcbb800ec915-kube-api-access-l2szl\") pod \"router-default-699696b566-qzknj\" (UID: \"9733272a-e52f-47ca-abc7-fcbb800ec915\") " pod="openshift-ingress/router-default-699696b566-qzknj" Sep 10 02:49:49.380486 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.380355 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9733272a-e52f-47ca-abc7-fcbb800ec915-metrics-certs\") pod \"router-default-699696b566-qzknj\" (UID: \"9733272a-e52f-47ca-abc7-fcbb800ec915\") " pod="openshift-ingress/router-default-699696b566-qzknj" Sep 10 02:49:49.380486 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.380395 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9733272a-e52f-47ca-abc7-fcbb800ec915-stats-auth\") pod \"router-default-699696b566-qzknj\" (UID: \"9733272a-e52f-47ca-abc7-fcbb800ec915\") " pod="openshift-ingress/router-default-699696b566-qzknj" Sep 10 02:49:49.380604 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.380483 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9733272a-e52f-47ca-abc7-fcbb800ec915-default-certificate\") pod \"router-default-699696b566-qzknj\" (UID: \"9733272a-e52f-47ca-abc7-fcbb800ec915\") " pod="openshift-ingress/router-default-699696b566-qzknj" Sep 10 02:49:49.380642 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.380602 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9733272a-e52f-47ca-abc7-fcbb800ec915-service-ca-bundle\") pod \"router-default-699696b566-qzknj\" (UID: \"9733272a-e52f-47ca-abc7-fcbb800ec915\") " pod="openshift-ingress/router-default-699696b566-qzknj" Sep 10 02:49:49.384358 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.384338 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-jxcwr"] Sep 10 02:49:49.384470 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.384452 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-l89pb" Sep 10 02:49:49.386807 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.386789 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-r82hd\"" Sep 10 02:49:49.402476 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.402446 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xqrx"] Sep 10 02:49:49.402616 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.402603 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-jxcwr" Sep 10 02:49:49.405052 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.405029 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Sep 10 02:49:49.405161 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.405100 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Sep 10 02:49:49.405365 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.405337 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 10 02:49:49.405468 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.405380 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-pmzdg\"" Sep 10 02:49:49.405468 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.405405 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 10 02:49:49.410893 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.410874 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Sep 10 02:49:49.424556 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.424527 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-6w544"] Sep 10 02:49:49.424681 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.424644 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xqrx" Sep 10 02:49:49.427656 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.427638 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Sep 10 02:49:49.428185 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.428170 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-xvq4l\"" Sep 10 02:49:49.428718 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.428704 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 10 02:49:49.429901 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.429883 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 10 02:49:49.429970 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.429916 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Sep 10 02:49:49.436668 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.436648 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-dvdwf"] Sep 10 02:49:49.436763 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.436683 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-l89pb"] Sep 10 02:49:49.436763 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.436704 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xqrx"] Sep 10 02:49:49.436763 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.436723 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-6w544"] Sep 10 02:49:49.436763 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.436746 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-jxcwr"] Sep 10 02:49:49.436763 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.436756 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-6w544" Sep 10 02:49:49.440171 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.439409 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-w9ph8"] Sep 10 02:49:49.441889 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.441865 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-688qk\"" Sep 10 02:49:49.442139 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.442122 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Sep 10 02:49:49.442352 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.442333 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Sep 10 02:49:49.442604 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.442132 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Sep 10 02:49:49.461651 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.461628 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-ghs46"] Sep 10 02:49:49.461767 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.461741 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-w9ph8" Sep 10 02:49:49.464259 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.464241 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Sep 10 02:49:49.464259 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.464249 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Sep 10 02:49:49.464404 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.464310 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-tlx2q\"" Sep 10 02:49:49.481245 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.481221 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9733272a-e52f-47ca-abc7-fcbb800ec915-stats-auth\") pod \"router-default-699696b566-qzknj\" (UID: \"9733272a-e52f-47ca-abc7-fcbb800ec915\") " pod="openshift-ingress/router-default-699696b566-qzknj" Sep 10 02:49:49.481381 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.481252 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d06965d1-f868-4a05-a4a8-1878d2d8ced1-config\") pod \"service-ca-operator-859f94c4b-dvdwf\" (UID: \"d06965d1-f868-4a05-a4a8-1878d2d8ced1\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-dvdwf" Sep 10 02:49:49.481381 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.481279 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9733272a-e52f-47ca-abc7-fcbb800ec915-default-certificate\") pod \"router-default-699696b566-qzknj\" (UID: \"9733272a-e52f-47ca-abc7-fcbb800ec915\") " pod="openshift-ingress/router-default-699696b566-qzknj" Sep 10 02:49:49.481381 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.481299 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d06965d1-f868-4a05-a4a8-1878d2d8ced1-serving-cert\") pod \"service-ca-operator-859f94c4b-dvdwf\" (UID: \"d06965d1-f868-4a05-a4a8-1878d2d8ced1\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-dvdwf" Sep 10 02:49:49.481589 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.481440 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9733272a-e52f-47ca-abc7-fcbb800ec915-service-ca-bundle\") pod \"router-default-699696b566-qzknj\" (UID: \"9733272a-e52f-47ca-abc7-fcbb800ec915\") " pod="openshift-ingress/router-default-699696b566-qzknj" Sep 10 02:49:49.481589 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.481460 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgbjk\" (UniqueName: \"kubernetes.io/projected/3b2ed542-ddab-42f4-8790-cf247106878b-kube-api-access-lgbjk\") pod \"network-check-source-685456655f-l89pb\" (UID: \"3b2ed542-ddab-42f4-8790-cf247106878b\") " pod="openshift-network-diagnostics/network-check-source-685456655f-l89pb" Sep 10 02:49:49.481589 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.481532 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2szl\" (UniqueName: \"kubernetes.io/projected/9733272a-e52f-47ca-abc7-fcbb800ec915-kube-api-access-l2szl\") pod \"router-default-699696b566-qzknj\" (UID: \"9733272a-e52f-47ca-abc7-fcbb800ec915\") " pod="openshift-ingress/router-default-699696b566-qzknj" Sep 10 02:49:49.481712 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.481585 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9733272a-e52f-47ca-abc7-fcbb800ec915-metrics-certs\") pod \"router-default-699696b566-qzknj\" (UID: \"9733272a-e52f-47ca-abc7-fcbb800ec915\") " pod="openshift-ingress/router-default-699696b566-qzknj" Sep 10 02:49:49.481712 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:49.481600 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9733272a-e52f-47ca-abc7-fcbb800ec915-service-ca-bundle podName:9733272a-e52f-47ca-abc7-fcbb800ec915 nodeName:}" failed. No retries permitted until 2026-09-10 02:49:49.981580183 +0000 UTC m=+48.828409451 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9733272a-e52f-47ca-abc7-fcbb800ec915-service-ca-bundle") pod "router-default-699696b566-qzknj" (UID: "9733272a-e52f-47ca-abc7-fcbb800ec915") : configmap references non-existent config key: service-ca.crt Sep 10 02:49:49.481712 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:49.481653 2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 10 02:49:49.481712 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.481653 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtmh8\" (UniqueName: \"kubernetes.io/projected/d06965d1-f868-4a05-a4a8-1878d2d8ced1-kube-api-access-vtmh8\") pod \"service-ca-operator-859f94c4b-dvdwf\" (UID: \"d06965d1-f868-4a05-a4a8-1878d2d8ced1\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-dvdwf" Sep 10 02:49:49.481929 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:49.481711 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9733272a-e52f-47ca-abc7-fcbb800ec915-metrics-certs podName:9733272a-e52f-47ca-abc7-fcbb800ec915 nodeName:}" failed. No retries permitted until 2026-09-10 02:49:49.981694988 +0000 UTC m=+48.828524252 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9733272a-e52f-47ca-abc7-fcbb800ec915-metrics-certs") pod "router-default-699696b566-qzknj" (UID: "9733272a-e52f-47ca-abc7-fcbb800ec915") : secret "router-metrics-certs-default" not found Sep 10 02:49:49.483711 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.483694 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-mpwt5"] Sep 10 02:49:49.483891 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.483867 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-ghs46" Sep 10 02:49:49.485042 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.485020 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9733272a-e52f-47ca-abc7-fcbb800ec915-default-certificate\") pod \"router-default-699696b566-qzknj\" (UID: \"9733272a-e52f-47ca-abc7-fcbb800ec915\") " pod="openshift-ingress/router-default-699696b566-qzknj" Sep 10 02:49:49.485142 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.485103 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9733272a-e52f-47ca-abc7-fcbb800ec915-stats-auth\") pod \"router-default-699696b566-qzknj\" (UID: \"9733272a-e52f-47ca-abc7-fcbb800ec915\") " pod="openshift-ingress/router-default-699696b566-qzknj" Sep 10 02:49:49.487746 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.487730 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Sep 10 02:49:49.487796 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.487751 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Sep 10 02:49:49.487995 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.487977 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Sep 10 02:49:49.488246 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.488231 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-zvh9t\"" Sep 10 02:49:49.489357 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.489332 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Sep 10 02:49:49.493095 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.493077 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Sep 10 02:49:49.496691 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.496671 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2szl\" (UniqueName: \"kubernetes.io/projected/9733272a-e52f-47ca-abc7-fcbb800ec915-kube-api-access-l2szl\") pod \"router-default-699696b566-qzknj\" (UID: \"9733272a-e52f-47ca-abc7-fcbb800ec915\") " pod="openshift-ingress/router-default-699696b566-qzknj" Sep 10 02:49:49.507912 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.507857 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-w9ph8"] Sep 10 02:49:49.507912 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.507893 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-ghs46"] Sep 10 02:49:49.507912 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.507907 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-mpwt5"] Sep 10 02:49:49.508056 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.507960 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-mpwt5" Sep 10 02:49:49.509987 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.509968 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Sep 10 02:49:49.510425 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.510405 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Sep 10 02:49:49.510544 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.510462 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-hsg6t\"" Sep 10 02:49:49.510544 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.510494 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Sep 10 02:49:49.510762 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.510746 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Sep 10 02:49:49.582288 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.582247 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtmh8\" (UniqueName: \"kubernetes.io/projected/d06965d1-f868-4a05-a4a8-1878d2d8ced1-kube-api-access-vtmh8\") pod \"service-ca-operator-859f94c4b-dvdwf\" (UID: \"d06965d1-f868-4a05-a4a8-1878d2d8ced1\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-dvdwf" Sep 10 02:49:49.582288 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.582293 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lm2d\" (UniqueName: \"kubernetes.io/projected/c2f7d2cb-e085-459b-904e-773fa7dad4e7-kube-api-access-8lm2d\") pod \"volume-data-source-validator-695c895bd6-w9ph8\" (UID: \"c2f7d2cb-e085-459b-904e-773fa7dad4e7\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-w9ph8" Sep 10 02:49:49.582497 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.582314 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d06965d1-f868-4a05-a4a8-1878d2d8ced1-config\") pod \"service-ca-operator-859f94c4b-dvdwf\" (UID: \"d06965d1-f868-4a05-a4a8-1878d2d8ced1\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-dvdwf" Sep 10 02:49:49.582497 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.582331 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gm2xv\" (UniqueName: \"kubernetes.io/projected/4d65d823-e75e-4f53-b9f1-c6a5ed728133-kube-api-access-gm2xv\") pod \"cluster-monitoring-operator-6cdcc96965-7xqrx\" (UID: \"4d65d823-e75e-4f53-b9f1-c6a5ed728133\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xqrx" Sep 10 02:49:49.582497 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.582392 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d06965d1-f868-4a05-a4a8-1878d2d8ced1-serving-cert\") pod \"service-ca-operator-859f94c4b-dvdwf\" (UID: \"d06965d1-f868-4a05-a4a8-1878d2d8ced1\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-dvdwf" Sep 10 02:49:49.582497 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.582413 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2badc582-316e-4b1e-b181-760ea2abbc08-tmp\") pod \"insights-operator-76bcd6ccf5-jxcwr\" (UID: \"2badc582-316e-4b1e-b181-760ea2abbc08\") " pod="openshift-insights/insights-operator-76bcd6ccf5-jxcwr" Sep 10 02:49:49.582497 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.582435 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2badc582-316e-4b1e-b181-760ea2abbc08-snapshots\") pod \"insights-operator-76bcd6ccf5-jxcwr\" (UID: \"2badc582-316e-4b1e-b181-760ea2abbc08\") " pod="openshift-insights/insights-operator-76bcd6ccf5-jxcwr" Sep 10 02:49:49.582497 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.582478 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a5210aa8-4171-4cc3-b65b-6237a9aee1d6-serving-cert\") pod \"console-operator-85b454874b-ghs46\" (UID: \"a5210aa8-4171-4cc3-b65b-6237a9aee1d6\") " pod="openshift-console-operator/console-operator-85b454874b-ghs46" Sep 10 02:49:49.582732 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.582553 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/49552b19-52f2-41d8-bc59-40bb3a992d6a-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-6w544\" (UID: \"49552b19-52f2-41d8-bc59-40bb3a992d6a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-6w544" Sep 10 02:49:49.582732 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.582594 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4d65d823-e75e-4f53-b9f1-c6a5ed728133-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-7xqrx\" (UID: \"4d65d823-e75e-4f53-b9f1-c6a5ed728133\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xqrx" Sep 10 02:49:49.582732 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.582669 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85sdw\" (UniqueName: \"kubernetes.io/projected/49552b19-52f2-41d8-bc59-40bb3a992d6a-kube-api-access-85sdw\") pod \"cluster-samples-operator-7cdcfcf76c-6w544\" (UID: \"49552b19-52f2-41d8-bc59-40bb3a992d6a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-6w544" Sep 10 02:49:49.582732 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.582710 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a5210aa8-4171-4cc3-b65b-6237a9aee1d6-trusted-ca\") pod \"console-operator-85b454874b-ghs46\" (UID: \"a5210aa8-4171-4cc3-b65b-6237a9aee1d6\") " pod="openshift-console-operator/console-operator-85b454874b-ghs46" Sep 10 02:49:49.582874 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.582735 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2badc582-316e-4b1e-b181-760ea2abbc08-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-jxcwr\" (UID: \"2badc582-316e-4b1e-b181-760ea2abbc08\") " pod="openshift-insights/insights-operator-76bcd6ccf5-jxcwr" Sep 10 02:49:49.582874 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.582788 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2badc582-316e-4b1e-b181-760ea2abbc08-serving-cert\") pod \"insights-operator-76bcd6ccf5-jxcwr\" (UID: \"2badc582-316e-4b1e-b181-760ea2abbc08\") " pod="openshift-insights/insights-operator-76bcd6ccf5-jxcwr" Sep 10 02:49:49.582874 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.582815 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgbjk\" (UniqueName: \"kubernetes.io/projected/3b2ed542-ddab-42f4-8790-cf247106878b-kube-api-access-lgbjk\") pod \"network-check-source-685456655f-l89pb\" (UID: \"3b2ed542-ddab-42f4-8790-cf247106878b\") " pod="openshift-network-diagnostics/network-check-source-685456655f-l89pb" Sep 10 02:49:49.582874 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.582850 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/4d65d823-e75e-4f53-b9f1-c6a5ed728133-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-7xqrx\" (UID: \"4d65d823-e75e-4f53-b9f1-c6a5ed728133\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xqrx" Sep 10 02:49:49.582874 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.582871 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2badc582-316e-4b1e-b181-760ea2abbc08-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-jxcwr\" (UID: \"2badc582-316e-4b1e-b181-760ea2abbc08\") " pod="openshift-insights/insights-operator-76bcd6ccf5-jxcwr" Sep 10 02:49:49.583057 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.582892 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bcd6\" (UniqueName: \"kubernetes.io/projected/2badc582-316e-4b1e-b181-760ea2abbc08-kube-api-access-4bcd6\") pod \"insights-operator-76bcd6ccf5-jxcwr\" (UID: \"2badc582-316e-4b1e-b181-760ea2abbc08\") " pod="openshift-insights/insights-operator-76bcd6ccf5-jxcwr" Sep 10 02:49:49.583057 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.582907 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a5210aa8-4171-4cc3-b65b-6237a9aee1d6-config\") pod \"console-operator-85b454874b-ghs46\" (UID: \"a5210aa8-4171-4cc3-b65b-6237a9aee1d6\") " pod="openshift-console-operator/console-operator-85b454874b-ghs46" Sep 10 02:49:49.583057 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.582929 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkwnp\" (UniqueName: \"kubernetes.io/projected/a5210aa8-4171-4cc3-b65b-6237a9aee1d6-kube-api-access-mkwnp\") pod \"console-operator-85b454874b-ghs46\" (UID: \"a5210aa8-4171-4cc3-b65b-6237a9aee1d6\") " pod="openshift-console-operator/console-operator-85b454874b-ghs46" Sep 10 02:49:49.583057 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.582968 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d06965d1-f868-4a05-a4a8-1878d2d8ced1-config\") pod \"service-ca-operator-859f94c4b-dvdwf\" (UID: \"d06965d1-f868-4a05-a4a8-1878d2d8ced1\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-dvdwf" Sep 10 02:49:49.584828 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.584812 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d06965d1-f868-4a05-a4a8-1878d2d8ced1-serving-cert\") pod \"service-ca-operator-859f94c4b-dvdwf\" (UID: \"d06965d1-f868-4a05-a4a8-1878d2d8ced1\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-dvdwf" Sep 10 02:49:49.593488 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.593464 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgbjk\" (UniqueName: \"kubernetes.io/projected/3b2ed542-ddab-42f4-8790-cf247106878b-kube-api-access-lgbjk\") pod \"network-check-source-685456655f-l89pb\" (UID: \"3b2ed542-ddab-42f4-8790-cf247106878b\") " pod="openshift-network-diagnostics/network-check-source-685456655f-l89pb" Sep 10 02:49:49.593711 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.593694 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtmh8\" (UniqueName: \"kubernetes.io/projected/d06965d1-f868-4a05-a4a8-1878d2d8ced1-kube-api-access-vtmh8\") pod \"service-ca-operator-859f94c4b-dvdwf\" (UID: \"d06965d1-f868-4a05-a4a8-1878d2d8ced1\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-dvdwf" Sep 10 02:49:49.670171 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.670134 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-dvdwf" Sep 10 02:49:49.684039 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.684007 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxz7d\" (UniqueName: \"kubernetes.io/projected/0a4e6681-5d69-48a7-b8e7-782f3e699b0a-kube-api-access-dxz7d\") pod \"kube-storage-version-migrator-operator-8d7dd4675-mpwt5\" (UID: \"0a4e6681-5d69-48a7-b8e7-782f3e699b0a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-mpwt5" Sep 10 02:49:49.684151 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.684051 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lm2d\" (UniqueName: \"kubernetes.io/projected/c2f7d2cb-e085-459b-904e-773fa7dad4e7-kube-api-access-8lm2d\") pod \"volume-data-source-validator-695c895bd6-w9ph8\" (UID: \"c2f7d2cb-e085-459b-904e-773fa7dad4e7\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-w9ph8" Sep 10 02:49:49.684151 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.684074 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gm2xv\" (UniqueName: \"kubernetes.io/projected/4d65d823-e75e-4f53-b9f1-c6a5ed728133-kube-api-access-gm2xv\") pod \"cluster-monitoring-operator-6cdcc96965-7xqrx\" (UID: \"4d65d823-e75e-4f53-b9f1-c6a5ed728133\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xqrx" Sep 10 02:49:49.684151 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.684122 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2badc582-316e-4b1e-b181-760ea2abbc08-tmp\") pod \"insights-operator-76bcd6ccf5-jxcwr\" (UID: \"2badc582-316e-4b1e-b181-760ea2abbc08\") " pod="openshift-insights/insights-operator-76bcd6ccf5-jxcwr" Sep 10 02:49:49.684151 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.684142 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2badc582-316e-4b1e-b181-760ea2abbc08-snapshots\") pod \"insights-operator-76bcd6ccf5-jxcwr\" (UID: \"2badc582-316e-4b1e-b181-760ea2abbc08\") " pod="openshift-insights/insights-operator-76bcd6ccf5-jxcwr" Sep 10 02:49:49.684347 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.684161 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a5210aa8-4171-4cc3-b65b-6237a9aee1d6-serving-cert\") pod \"console-operator-85b454874b-ghs46\" (UID: \"a5210aa8-4171-4cc3-b65b-6237a9aee1d6\") " pod="openshift-console-operator/console-operator-85b454874b-ghs46" Sep 10 02:49:49.684347 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.684185 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0a4e6681-5d69-48a7-b8e7-782f3e699b0a-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-mpwt5\" (UID: \"0a4e6681-5d69-48a7-b8e7-782f3e699b0a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-mpwt5" Sep 10 02:49:49.684347 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.684233 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/49552b19-52f2-41d8-bc59-40bb3a992d6a-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-6w544\" (UID: \"49552b19-52f2-41d8-bc59-40bb3a992d6a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-6w544" Sep 10 02:49:49.684347 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.684258 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4d65d823-e75e-4f53-b9f1-c6a5ed728133-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-7xqrx\" (UID: \"4d65d823-e75e-4f53-b9f1-c6a5ed728133\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xqrx" Sep 10 02:49:49.684347 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.684297 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85sdw\" (UniqueName: \"kubernetes.io/projected/49552b19-52f2-41d8-bc59-40bb3a992d6a-kube-api-access-85sdw\") pod \"cluster-samples-operator-7cdcfcf76c-6w544\" (UID: \"49552b19-52f2-41d8-bc59-40bb3a992d6a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-6w544" Sep 10 02:49:49.684347 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.684329 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a5210aa8-4171-4cc3-b65b-6237a9aee1d6-trusted-ca\") pod \"console-operator-85b454874b-ghs46\" (UID: \"a5210aa8-4171-4cc3-b65b-6237a9aee1d6\") " pod="openshift-console-operator/console-operator-85b454874b-ghs46" Sep 10 02:49:49.684652 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.684362 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2badc582-316e-4b1e-b181-760ea2abbc08-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-jxcwr\" (UID: \"2badc582-316e-4b1e-b181-760ea2abbc08\") " pod="openshift-insights/insights-operator-76bcd6ccf5-jxcwr" Sep 10 02:49:49.684652 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:49.684371 2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 02:49:49.684652 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:49.684397 2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 02:49:49.684652 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.684404 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2badc582-316e-4b1e-b181-760ea2abbc08-serving-cert\") pod \"insights-operator-76bcd6ccf5-jxcwr\" (UID: \"2badc582-316e-4b1e-b181-760ea2abbc08\") " pod="openshift-insights/insights-operator-76bcd6ccf5-jxcwr" Sep 10 02:49:49.684652 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:49.684437 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d65d823-e75e-4f53-b9f1-c6a5ed728133-cluster-monitoring-operator-tls podName:4d65d823-e75e-4f53-b9f1-c6a5ed728133 nodeName:}" failed. No retries permitted until 2026-09-10 02:49:50.18441643 +0000 UTC m=+49.031245694 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4d65d823-e75e-4f53-b9f1-c6a5ed728133-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-7xqrx" (UID: "4d65d823-e75e-4f53-b9f1-c6a5ed728133") : secret "cluster-monitoring-operator-tls" not found Sep 10 02:49:49.684652 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.684474 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/4d65d823-e75e-4f53-b9f1-c6a5ed728133-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-7xqrx\" (UID: \"4d65d823-e75e-4f53-b9f1-c6a5ed728133\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xqrx" Sep 10 02:49:49.684652 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.684488 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2badc582-316e-4b1e-b181-760ea2abbc08-tmp\") pod \"insights-operator-76bcd6ccf5-jxcwr\" (UID: \"2badc582-316e-4b1e-b181-760ea2abbc08\") " pod="openshift-insights/insights-operator-76bcd6ccf5-jxcwr" Sep 10 02:49:49.684652 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:49.684536 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49552b19-52f2-41d8-bc59-40bb3a992d6a-samples-operator-tls podName:49552b19-52f2-41d8-bc59-40bb3a992d6a nodeName:}" failed. No retries permitted until 2026-09-10 02:49:50.184495834 +0000 UTC m=+49.031325100 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/49552b19-52f2-41d8-bc59-40bb3a992d6a-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-6w544" (UID: "49552b19-52f2-41d8-bc59-40bb3a992d6a") : secret "samples-operator-tls" not found Sep 10 02:49:49.684652 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.684580 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2badc582-316e-4b1e-b181-760ea2abbc08-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-jxcwr\" (UID: \"2badc582-316e-4b1e-b181-760ea2abbc08\") " pod="openshift-insights/insights-operator-76bcd6ccf5-jxcwr" Sep 10 02:49:49.684652 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.684622 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bcd6\" (UniqueName: \"kubernetes.io/projected/2badc582-316e-4b1e-b181-760ea2abbc08-kube-api-access-4bcd6\") pod \"insights-operator-76bcd6ccf5-jxcwr\" (UID: \"2badc582-316e-4b1e-b181-760ea2abbc08\") " pod="openshift-insights/insights-operator-76bcd6ccf5-jxcwr" Sep 10 02:49:49.685129 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.684684 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0a4e6681-5d69-48a7-b8e7-782f3e699b0a-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-mpwt5\" (UID: \"0a4e6681-5d69-48a7-b8e7-782f3e699b0a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-mpwt5" Sep 10 02:49:49.685129 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.684714 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a5210aa8-4171-4cc3-b65b-6237a9aee1d6-config\") pod \"console-operator-85b454874b-ghs46\" (UID: \"a5210aa8-4171-4cc3-b65b-6237a9aee1d6\") " pod="openshift-console-operator/console-operator-85b454874b-ghs46" Sep 10 02:49:49.685129 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.684752 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkwnp\" (UniqueName: \"kubernetes.io/projected/a5210aa8-4171-4cc3-b65b-6237a9aee1d6-kube-api-access-mkwnp\") pod \"console-operator-85b454874b-ghs46\" (UID: \"a5210aa8-4171-4cc3-b65b-6237a9aee1d6\") " pod="openshift-console-operator/console-operator-85b454874b-ghs46" Sep 10 02:49:49.685343 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.685226 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2badc582-316e-4b1e-b181-760ea2abbc08-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-jxcwr\" (UID: \"2badc582-316e-4b1e-b181-760ea2abbc08\") " pod="openshift-insights/insights-operator-76bcd6ccf5-jxcwr" Sep 10 02:49:49.685343 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.685289 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/4d65d823-e75e-4f53-b9f1-c6a5ed728133-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-7xqrx\" (UID: \"4d65d823-e75e-4f53-b9f1-c6a5ed728133\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xqrx" Sep 10 02:49:49.685487 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.685421 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a5210aa8-4171-4cc3-b65b-6237a9aee1d6-trusted-ca\") pod \"console-operator-85b454874b-ghs46\" (UID: \"a5210aa8-4171-4cc3-b65b-6237a9aee1d6\") " pod="openshift-console-operator/console-operator-85b454874b-ghs46" Sep 10 02:49:49.685487 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.685432 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2badc582-316e-4b1e-b181-760ea2abbc08-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-jxcwr\" (UID: \"2badc582-316e-4b1e-b181-760ea2abbc08\") " pod="openshift-insights/insights-operator-76bcd6ccf5-jxcwr" Sep 10 02:49:49.685882 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.685862 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a5210aa8-4171-4cc3-b65b-6237a9aee1d6-config\") pod \"console-operator-85b454874b-ghs46\" (UID: \"a5210aa8-4171-4cc3-b65b-6237a9aee1d6\") " pod="openshift-console-operator/console-operator-85b454874b-ghs46" Sep 10 02:49:49.686001 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.685980 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2badc582-316e-4b1e-b181-760ea2abbc08-snapshots\") pod \"insights-operator-76bcd6ccf5-jxcwr\" (UID: \"2badc582-316e-4b1e-b181-760ea2abbc08\") " pod="openshift-insights/insights-operator-76bcd6ccf5-jxcwr" Sep 10 02:49:49.686863 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.686848 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a5210aa8-4171-4cc3-b65b-6237a9aee1d6-serving-cert\") pod \"console-operator-85b454874b-ghs46\" (UID: \"a5210aa8-4171-4cc3-b65b-6237a9aee1d6\") " pod="openshift-console-operator/console-operator-85b454874b-ghs46" Sep 10 02:49:49.686912 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.686895 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2badc582-316e-4b1e-b181-760ea2abbc08-serving-cert\") pod \"insights-operator-76bcd6ccf5-jxcwr\" (UID: \"2badc582-316e-4b1e-b181-760ea2abbc08\") " pod="openshift-insights/insights-operator-76bcd6ccf5-jxcwr" Sep 10 02:49:49.692564 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.692246 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-l89pb" Sep 10 02:49:49.694895 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.694871 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkwnp\" (UniqueName: \"kubernetes.io/projected/a5210aa8-4171-4cc3-b65b-6237a9aee1d6-kube-api-access-mkwnp\") pod \"console-operator-85b454874b-ghs46\" (UID: \"a5210aa8-4171-4cc3-b65b-6237a9aee1d6\") " pod="openshift-console-operator/console-operator-85b454874b-ghs46" Sep 10 02:49:49.695266 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.695241 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85sdw\" (UniqueName: \"kubernetes.io/projected/49552b19-52f2-41d8-bc59-40bb3a992d6a-kube-api-access-85sdw\") pod \"cluster-samples-operator-7cdcfcf76c-6w544\" (UID: \"49552b19-52f2-41d8-bc59-40bb3a992d6a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-6w544" Sep 10 02:49:49.695433 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.695415 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lm2d\" (UniqueName: \"kubernetes.io/projected/c2f7d2cb-e085-459b-904e-773fa7dad4e7-kube-api-access-8lm2d\") pod \"volume-data-source-validator-695c895bd6-w9ph8\" (UID: \"c2f7d2cb-e085-459b-904e-773fa7dad4e7\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-w9ph8" Sep 10 02:49:49.695619 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.695598 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gm2xv\" (UniqueName: \"kubernetes.io/projected/4d65d823-e75e-4f53-b9f1-c6a5ed728133-kube-api-access-gm2xv\") pod \"cluster-monitoring-operator-6cdcc96965-7xqrx\" (UID: \"4d65d823-e75e-4f53-b9f1-c6a5ed728133\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xqrx" Sep 10 02:49:49.695975 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.695956 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bcd6\" (UniqueName: \"kubernetes.io/projected/2badc582-316e-4b1e-b181-760ea2abbc08-kube-api-access-4bcd6\") pod \"insights-operator-76bcd6ccf5-jxcwr\" (UID: \"2badc582-316e-4b1e-b181-760ea2abbc08\") " pod="openshift-insights/insights-operator-76bcd6ccf5-jxcwr" Sep 10 02:49:49.712040 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.712019 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-jxcwr" Sep 10 02:49:49.771421 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.771394 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-w9ph8" Sep 10 02:49:49.786126 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.786096 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0a4e6681-5d69-48a7-b8e7-782f3e699b0a-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-mpwt5\" (UID: \"0a4e6681-5d69-48a7-b8e7-782f3e699b0a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-mpwt5" Sep 10 02:49:49.786275 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.786148 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxz7d\" (UniqueName: \"kubernetes.io/projected/0a4e6681-5d69-48a7-b8e7-782f3e699b0a-kube-api-access-dxz7d\") pod \"kube-storage-version-migrator-operator-8d7dd4675-mpwt5\" (UID: \"0a4e6681-5d69-48a7-b8e7-782f3e699b0a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-mpwt5" Sep 10 02:49:49.786368 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.786347 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0a4e6681-5d69-48a7-b8e7-782f3e699b0a-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-mpwt5\" (UID: \"0a4e6681-5d69-48a7-b8e7-782f3e699b0a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-mpwt5" Sep 10 02:49:49.786881 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.786860 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0a4e6681-5d69-48a7-b8e7-782f3e699b0a-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-mpwt5\" (UID: \"0a4e6681-5d69-48a7-b8e7-782f3e699b0a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-mpwt5" Sep 10 02:49:49.788696 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.788653 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0a4e6681-5d69-48a7-b8e7-782f3e699b0a-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-mpwt5\" (UID: \"0a4e6681-5d69-48a7-b8e7-782f3e699b0a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-mpwt5" Sep 10 02:49:49.799341 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.799241 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxz7d\" (UniqueName: \"kubernetes.io/projected/0a4e6681-5d69-48a7-b8e7-782f3e699b0a-kube-api-access-dxz7d\") pod \"kube-storage-version-migrator-operator-8d7dd4675-mpwt5\" (UID: \"0a4e6681-5d69-48a7-b8e7-782f3e699b0a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-mpwt5" Sep 10 02:49:49.803545 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.802964 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-ghs46" Sep 10 02:49:49.815949 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.815921 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-mpwt5" Sep 10 02:49:49.969839 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.969810 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-dvdwf"] Sep 10 02:49:49.973118 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:49.973091 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd06965d1_f868_4a05_a4a8_1878d2d8ced1.slice/crio-b6fad5794926a090a6b807dfc9741fe86e589c10a8aa59e9f6fc71954e030f55 WatchSource:0}: Error finding container b6fad5794926a090a6b807dfc9741fe86e589c10a8aa59e9f6fc71954e030f55: Status 404 returned error can't find the container with id b6fad5794926a090a6b807dfc9741fe86e589c10a8aa59e9f6fc71954e030f55 Sep 10 02:49:49.988197 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.988175 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9733272a-e52f-47ca-abc7-fcbb800ec915-service-ca-bundle\") pod \"router-default-699696b566-qzknj\" (UID: \"9733272a-e52f-47ca-abc7-fcbb800ec915\") " pod="openshift-ingress/router-default-699696b566-qzknj" Sep 10 02:49:49.988309 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.988234 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9733272a-e52f-47ca-abc7-fcbb800ec915-metrics-certs\") pod \"router-default-699696b566-qzknj\" (UID: \"9733272a-e52f-47ca-abc7-fcbb800ec915\") " pod="openshift-ingress/router-default-699696b566-qzknj" Sep 10 02:49:49.988399 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:49.988382 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9733272a-e52f-47ca-abc7-fcbb800ec915-service-ca-bundle podName:9733272a-e52f-47ca-abc7-fcbb800ec915 nodeName:}" failed. No retries permitted until 2026-09-10 02:49:50.988358154 +0000 UTC m=+49.835187421 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9733272a-e52f-47ca-abc7-fcbb800ec915-service-ca-bundle") pod "router-default-699696b566-qzknj" (UID: "9733272a-e52f-47ca-abc7-fcbb800ec915") : configmap references non-existent config key: service-ca.crt Sep 10 02:49:49.988399 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:49.988390 2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 10 02:49:49.988525 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:49.988445 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9733272a-e52f-47ca-abc7-fcbb800ec915-metrics-certs podName:9733272a-e52f-47ca-abc7-fcbb800ec915 nodeName:}" failed. No retries permitted until 2026-09-10 02:49:50.98842916 +0000 UTC m=+49.835258426 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9733272a-e52f-47ca-abc7-fcbb800ec915-metrics-certs") pod "router-default-699696b566-qzknj" (UID: "9733272a-e52f-47ca-abc7-fcbb800ec915") : secret "router-metrics-certs-default" not found Sep 10 02:49:49.996026 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:49.996007 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-l89pb"] Sep 10 02:49:49.998372 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:49.998352 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b2ed542_ddab_42f4_8790_cf247106878b.slice/crio-b3d2fde4636c595cbfe88ec2f19976f7d1dcf6c02d6d3e41e61b680763fc7868 WatchSource:0}: Error finding container b3d2fde4636c595cbfe88ec2f19976f7d1dcf6c02d6d3e41e61b680763fc7868: Status 404 returned error can't find the container with id b3d2fde4636c595cbfe88ec2f19976f7d1dcf6c02d6d3e41e61b680763fc7868 Sep 10 02:49:50.039182 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:50.037520 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-jxcwr"] Sep 10 02:49:50.039663 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:50.039636 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2badc582_316e_4b1e_b181_760ea2abbc08.slice/crio-d7d918829a514c3c52066caaf63e1e97617e9dabfbf1b6be6792504cfee8be1b WatchSource:0}: Error finding container d7d918829a514c3c52066caaf63e1e97617e9dabfbf1b6be6792504cfee8be1b: Status 404 returned error can't find the container with id d7d918829a514c3c52066caaf63e1e97617e9dabfbf1b6be6792504cfee8be1b Sep 10 02:49:50.082329 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:50.082211 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-w9ph8"] Sep 10 02:49:50.084748 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:50.084705 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2f7d2cb_e085_459b_904e_773fa7dad4e7.slice/crio-e5da6b87ba0d3f7b06c66ff3569f45ca7e433d53b9beda78d6083ea43ce3f1b6 WatchSource:0}: Error finding container e5da6b87ba0d3f7b06c66ff3569f45ca7e433d53b9beda78d6083ea43ce3f1b6: Status 404 returned error can't find the container with id e5da6b87ba0d3f7b06c66ff3569f45ca7e433d53b9beda78d6083ea43ce3f1b6 Sep 10 02:49:50.104968 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:50.104940 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-ghs46"] Sep 10 02:49:50.107965 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:50.107939 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5210aa8_4171_4cc3_b65b_6237a9aee1d6.slice/crio-11b4cffbc87409f9b78733e67c905499e99fb72c50384536aca715c07a4c66fc WatchSource:0}: Error finding container 11b4cffbc87409f9b78733e67c905499e99fb72c50384536aca715c07a4c66fc: Status 404 returned error can't find the container with id 11b4cffbc87409f9b78733e67c905499e99fb72c50384536aca715c07a4c66fc Sep 10 02:49:50.127614 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:50.127578 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-mpwt5"] Sep 10 02:49:50.130412 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:49:50.130387 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a4e6681_5d69_48a7_b8e7_782f3e699b0a.slice/crio-caa35029718a7cd4649fbb51e59bc0243bda5dc9ea1c68253ec3accaa658524f WatchSource:0}: Error finding container caa35029718a7cd4649fbb51e59bc0243bda5dc9ea1c68253ec3accaa658524f: Status 404 returned error can't find the container with id caa35029718a7cd4649fbb51e59bc0243bda5dc9ea1c68253ec3accaa658524f Sep 10 02:49:50.190472 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:50.190432 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/49552b19-52f2-41d8-bc59-40bb3a992d6a-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-6w544\" (UID: \"49552b19-52f2-41d8-bc59-40bb3a992d6a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-6w544" Sep 10 02:49:50.190472 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:50.190479 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4d65d823-e75e-4f53-b9f1-c6a5ed728133-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-7xqrx\" (UID: \"4d65d823-e75e-4f53-b9f1-c6a5ed728133\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xqrx" Sep 10 02:49:50.190725 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:50.190594 2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 02:49:50.190725 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:50.190607 2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 02:49:50.190725 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:50.190658 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d65d823-e75e-4f53-b9f1-c6a5ed728133-cluster-monitoring-operator-tls podName:4d65d823-e75e-4f53-b9f1-c6a5ed728133 nodeName:}" failed. No retries permitted until 2026-09-10 02:49:51.190642711 +0000 UTC m=+50.037471974 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4d65d823-e75e-4f53-b9f1-c6a5ed728133-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-7xqrx" (UID: "4d65d823-e75e-4f53-b9f1-c6a5ed728133") : secret "cluster-monitoring-operator-tls" not found Sep 10 02:49:50.190725 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:50.190703 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49552b19-52f2-41d8-bc59-40bb3a992d6a-samples-operator-tls podName:49552b19-52f2-41d8-bc59-40bb3a992d6a nodeName:}" failed. No retries permitted until 2026-09-10 02:49:51.190693724 +0000 UTC m=+50.037522970 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/49552b19-52f2-41d8-bc59-40bb3a992d6a-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-6w544" (UID: "49552b19-52f2-41d8-bc59-40bb3a992d6a") : secret "samples-operator-tls" not found Sep 10 02:49:50.872316 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:50.872242 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-ghs46" event={"ID":"a5210aa8-4171-4cc3-b65b-6237a9aee1d6","Type":"ContainerStarted","Data":"11b4cffbc87409f9b78733e67c905499e99fb72c50384536aca715c07a4c66fc"} Sep 10 02:49:50.873999 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:50.873923 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-w9ph8" event={"ID":"c2f7d2cb-e085-459b-904e-773fa7dad4e7","Type":"ContainerStarted","Data":"e5da6b87ba0d3f7b06c66ff3569f45ca7e433d53b9beda78d6083ea43ce3f1b6"} Sep 10 02:49:50.876817 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:50.876737 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-jxcwr" event={"ID":"2badc582-316e-4b1e-b181-760ea2abbc08","Type":"ContainerStarted","Data":"d7d918829a514c3c52066caaf63e1e97617e9dabfbf1b6be6792504cfee8be1b"} Sep 10 02:49:50.878281 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:50.878204 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-l89pb" event={"ID":"3b2ed542-ddab-42f4-8790-cf247106878b","Type":"ContainerStarted","Data":"b3d2fde4636c595cbfe88ec2f19976f7d1dcf6c02d6d3e41e61b680763fc7868"} Sep 10 02:49:50.880363 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:50.880290 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-mpwt5" event={"ID":"0a4e6681-5d69-48a7-b8e7-782f3e699b0a","Type":"ContainerStarted","Data":"caa35029718a7cd4649fbb51e59bc0243bda5dc9ea1c68253ec3accaa658524f"} Sep 10 02:49:50.882140 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:50.882076 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-dvdwf" event={"ID":"d06965d1-f868-4a05-a4a8-1878d2d8ced1","Type":"ContainerStarted","Data":"b6fad5794926a090a6b807dfc9741fe86e589c10a8aa59e9f6fc71954e030f55"} Sep 10 02:49:51.000035 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:50.999289 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9733272a-e52f-47ca-abc7-fcbb800ec915-service-ca-bundle\") pod \"router-default-699696b566-qzknj\" (UID: \"9733272a-e52f-47ca-abc7-fcbb800ec915\") " pod="openshift-ingress/router-default-699696b566-qzknj" Sep 10 02:49:51.000035 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:50.999388 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9733272a-e52f-47ca-abc7-fcbb800ec915-metrics-certs\") pod \"router-default-699696b566-qzknj\" (UID: \"9733272a-e52f-47ca-abc7-fcbb800ec915\") " pod="openshift-ingress/router-default-699696b566-qzknj" Sep 10 02:49:51.000035 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:50.999537 2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 10 02:49:51.000035 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:50.999604 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9733272a-e52f-47ca-abc7-fcbb800ec915-metrics-certs podName:9733272a-e52f-47ca-abc7-fcbb800ec915 nodeName:}" failed. No retries permitted until 2026-09-10 02:49:52.999583624 +0000 UTC m=+51.846412872 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9733272a-e52f-47ca-abc7-fcbb800ec915-metrics-certs") pod "router-default-699696b566-qzknj" (UID: "9733272a-e52f-47ca-abc7-fcbb800ec915") : secret "router-metrics-certs-default" not found Sep 10 02:49:51.000035 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:50.999686 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9733272a-e52f-47ca-abc7-fcbb800ec915-service-ca-bundle podName:9733272a-e52f-47ca-abc7-fcbb800ec915 nodeName:}" failed. No retries permitted until 2026-09-10 02:49:52.999673397 +0000 UTC m=+51.846502660 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9733272a-e52f-47ca-abc7-fcbb800ec915-service-ca-bundle") pod "router-default-699696b566-qzknj" (UID: "9733272a-e52f-47ca-abc7-fcbb800ec915") : configmap references non-existent config key: service-ca.crt Sep 10 02:49:51.202264 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:51.202182 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/49552b19-52f2-41d8-bc59-40bb3a992d6a-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-6w544\" (UID: \"49552b19-52f2-41d8-bc59-40bb3a992d6a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-6w544" Sep 10 02:49:51.202264 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:51.202256 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4d65d823-e75e-4f53-b9f1-c6a5ed728133-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-7xqrx\" (UID: \"4d65d823-e75e-4f53-b9f1-c6a5ed728133\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xqrx" Sep 10 02:49:51.202484 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:51.202466 2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 02:49:51.202585 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:51.202572 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d65d823-e75e-4f53-b9f1-c6a5ed728133-cluster-monitoring-operator-tls podName:4d65d823-e75e-4f53-b9f1-c6a5ed728133 nodeName:}" failed. No retries permitted until 2026-09-10 02:49:53.202546728 +0000 UTC m=+52.049375997 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4d65d823-e75e-4f53-b9f1-c6a5ed728133-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-7xqrx" (UID: "4d65d823-e75e-4f53-b9f1-c6a5ed728133") : secret "cluster-monitoring-operator-tls" not found Sep 10 02:49:51.202872 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:51.202854 2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 02:49:51.202944 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:51.202909 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49552b19-52f2-41d8-bc59-40bb3a992d6a-samples-operator-tls podName:49552b19-52f2-41d8-bc59-40bb3a992d6a nodeName:}" failed. No retries permitted until 2026-09-10 02:49:53.202894145 +0000 UTC m=+52.049723408 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/49552b19-52f2-41d8-bc59-40bb3a992d6a-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-6w544" (UID: "49552b19-52f2-41d8-bc59-40bb3a992d6a") : secret "samples-operator-tls" not found Sep 10 02:49:51.910799 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:51.910756 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/98643336-46ce-4f17-acb2-d8110a5bfb84-metrics-tls\") pod \"dns-default-2s6nr\" (UID: \"98643336-46ce-4f17-acb2-d8110a5bfb84\") " pod="openshift-dns/dns-default-2s6nr" Sep 10 02:49:51.911305 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:51.910830 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-registry-tls\") pod \"image-registry-75f8b47547-m9np9\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " pod="openshift-image-registry/image-registry-75f8b47547-m9np9" Sep 10 02:49:51.911305 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:51.910995 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 02:49:51.911305 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:51.911011 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75f8b47547-m9np9: secret "image-registry-tls" not found Sep 10 02:49:51.911305 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:51.911067 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-registry-tls podName:e4d56e77-0fdc-4733-b06f-2451d59f8343 nodeName:}" failed. No retries permitted until 2026-09-10 02:50:07.911053143 +0000 UTC m=+66.757882402 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-registry-tls") pod "image-registry-75f8b47547-m9np9" (UID: "e4d56e77-0fdc-4733-b06f-2451d59f8343") : secret "image-registry-tls" not found Sep 10 02:49:51.911542 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:51.911438 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 02:49:51.911542 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:51.911475 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98643336-46ce-4f17-acb2-d8110a5bfb84-metrics-tls podName:98643336-46ce-4f17-acb2-d8110a5bfb84 nodeName:}" failed. No retries permitted until 2026-09-10 02:50:07.91146468 +0000 UTC m=+66.758293939 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/98643336-46ce-4f17-acb2-d8110a5bfb84-metrics-tls") pod "dns-default-2s6nr" (UID: "98643336-46ce-4f17-acb2-d8110a5bfb84") : secret "dns-default-metrics-tls" not found Sep 10 02:49:52.890406 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:52.890367 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-w9ph8" event={"ID":"c2f7d2cb-e085-459b-904e-773fa7dad4e7","Type":"ContainerStarted","Data":"b5ab4c1b2637a44faab9c2d1eeedfd9acf07039dc21d1e7cecdfe07321c2e9cc"} Sep 10 02:49:52.913555 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:52.912377 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-w9ph8" podStartSLOduration=1.9041423339999999 podStartE2EDuration="3.912350625s" totalImagesPullingTime="2.008208291s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 02:49:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:49:52.912252239 +0000 UTC m=+51.759081508" watchObservedRunningTime="2026-09-10 02:49:52.912350625 +0000 UTC m=+51.759179895" Sep 10 02:49:53.019336 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:53.019293 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9733272a-e52f-47ca-abc7-fcbb800ec915-service-ca-bundle\") pod \"router-default-699696b566-qzknj\" (UID: \"9733272a-e52f-47ca-abc7-fcbb800ec915\") " pod="openshift-ingress/router-default-699696b566-qzknj" Sep 10 02:49:53.019547 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:53.019392 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9733272a-e52f-47ca-abc7-fcbb800ec915-metrics-certs\") pod \"router-default-699696b566-qzknj\" (UID: \"9733272a-e52f-47ca-abc7-fcbb800ec915\") " pod="openshift-ingress/router-default-699696b566-qzknj" Sep 10 02:49:53.019547 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:53.019496 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9733272a-e52f-47ca-abc7-fcbb800ec915-service-ca-bundle podName:9733272a-e52f-47ca-abc7-fcbb800ec915 nodeName:}" failed. No retries permitted until 2026-09-10 02:49:57.019471717 +0000 UTC m=+55.866300976 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9733272a-e52f-47ca-abc7-fcbb800ec915-service-ca-bundle") pod "router-default-699696b566-qzknj" (UID: "9733272a-e52f-47ca-abc7-fcbb800ec915") : configmap references non-existent config key: service-ca.crt Sep 10 02:49:53.019686 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:53.019579 2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 10 02:49:53.019686 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:53.019645 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9733272a-e52f-47ca-abc7-fcbb800ec915-metrics-certs podName:9733272a-e52f-47ca-abc7-fcbb800ec915 nodeName:}" failed. No retries permitted until 2026-09-10 02:49:57.019627 +0000 UTC m=+55.866456268 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9733272a-e52f-47ca-abc7-fcbb800ec915-metrics-certs") pod "router-default-699696b566-qzknj" (UID: "9733272a-e52f-47ca-abc7-fcbb800ec915") : secret "router-metrics-certs-default" not found Sep 10 02:49:53.221649 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:53.221564 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/49552b19-52f2-41d8-bc59-40bb3a992d6a-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-6w544\" (UID: \"49552b19-52f2-41d8-bc59-40bb3a992d6a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-6w544" Sep 10 02:49:53.221649 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:53.221630 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4d65d823-e75e-4f53-b9f1-c6a5ed728133-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-7xqrx\" (UID: \"4d65d823-e75e-4f53-b9f1-c6a5ed728133\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xqrx" Sep 10 02:49:53.221876 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:53.221723 2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 02:49:53.221876 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:53.221772 2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 02:49:53.221876 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:53.221802 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49552b19-52f2-41d8-bc59-40bb3a992d6a-samples-operator-tls podName:49552b19-52f2-41d8-bc59-40bb3a992d6a nodeName:}" failed. No retries permitted until 2026-09-10 02:49:57.221781698 +0000 UTC m=+56.068610961 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/49552b19-52f2-41d8-bc59-40bb3a992d6a-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-6w544" (UID: "49552b19-52f2-41d8-bc59-40bb3a992d6a") : secret "samples-operator-tls" not found Sep 10 02:49:53.221876 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:53.221821 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d65d823-e75e-4f53-b9f1-c6a5ed728133-cluster-monitoring-operator-tls podName:4d65d823-e75e-4f53-b9f1-c6a5ed728133 nodeName:}" failed. No retries permitted until 2026-09-10 02:49:57.22181148 +0000 UTC m=+56.068640727 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4d65d823-e75e-4f53-b9f1-c6a5ed728133-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-7xqrx" (UID: "4d65d823-e75e-4f53-b9f1-c6a5ed728133") : secret "cluster-monitoring-operator-tls" not found Sep 10 02:49:56.904240 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:56.904204 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ghs46_a5210aa8-4171-4cc3-b65b-6237a9aee1d6/console-operator/0.log" Sep 10 02:49:56.904715 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:56.904258 2560 generic.go:356] "Generic (PLEG): container finished" podID="a5210aa8-4171-4cc3-b65b-6237a9aee1d6" containerID="54d3b4804f8d845b6fef4cd75a4fbddd1e1add7847575444590ee0ec2ec0ff87" exitCode=255 Sep 10 02:49:56.904715 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:56.904324 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-ghs46" event={"ID":"a5210aa8-4171-4cc3-b65b-6237a9aee1d6","Type":"ContainerDied","Data":"54d3b4804f8d845b6fef4cd75a4fbddd1e1add7847575444590ee0ec2ec0ff87"} Sep 10 02:49:56.904715 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:56.904625 2560 scope.go:117] "RemoveContainer" containerID="54d3b4804f8d845b6fef4cd75a4fbddd1e1add7847575444590ee0ec2ec0ff87" Sep 10 02:49:56.906385 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:56.906354 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-jxcwr" event={"ID":"2badc582-316e-4b1e-b181-760ea2abbc08","Type":"ContainerStarted","Data":"43bf28409b57d912c132e06aac81b5665de04b8dd157e290be36282b8083c0e7"} Sep 10 02:49:56.909072 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:56.908723 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-l89pb" event={"ID":"3b2ed542-ddab-42f4-8790-cf247106878b","Type":"ContainerStarted","Data":"a70432702cc4731904be040d4572840a3778168075b6e038be0e70930971ae32"} Sep 10 02:49:56.910427 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:56.910407 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-mpwt5" event={"ID":"0a4e6681-5d69-48a7-b8e7-782f3e699b0a","Type":"ContainerStarted","Data":"a99d3e705a6dc157e49ab7d16802f1599b3812a07fdda2c9c15d492791e0ef39"} Sep 10 02:49:56.912038 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:56.912017 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-dvdwf" event={"ID":"d06965d1-f868-4a05-a4a8-1878d2d8ced1","Type":"ContainerStarted","Data":"5be094f307d05149ea8401917ad20f920a790a6613e45dc9c420ddc8a6f20d5a"} Sep 10 02:49:56.939754 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:56.939698 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-dvdwf" podStartSLOduration=1.329942707 podStartE2EDuration="7.93967616s" totalImagesPullingTime="6.609733453s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 02:49:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:49:56.938066738 +0000 UTC m=+55.784896007" watchObservedRunningTime="2026-09-10 02:49:56.93967616 +0000 UTC m=+55.786505429" Sep 10 02:49:56.996360 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:56.995698 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-l89pb" podStartSLOduration=1.401459997 podStartE2EDuration="7.995677408s" totalImagesPullingTime="6.594217411s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 02:49:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:49:56.993722282 +0000 UTC m=+55.840551556" watchObservedRunningTime="2026-09-10 02:49:56.995677408 +0000 UTC m=+55.842506677" Sep 10 02:49:56.996360 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:56.996324 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-jxcwr" podStartSLOduration=1.45304984 podStartE2EDuration="7.996315083s" totalImagesPullingTime="6.543265243s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 02:49:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:49:56.96507636 +0000 UTC m=+55.811905629" watchObservedRunningTime="2026-09-10 02:49:56.996315083 +0000 UTC m=+55.843144352" Sep 10 02:49:57.018683 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:57.018616 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-mpwt5" podStartSLOduration=1.565708235 podStartE2EDuration="8.018596415s" totalImagesPullingTime="6.45288818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 02:49:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:49:57.016055054 +0000 UTC m=+55.862884323" watchObservedRunningTime="2026-09-10 02:49:57.018596415 +0000 UTC m=+55.865425697" Sep 10 02:49:57.058526 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:57.058226 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9733272a-e52f-47ca-abc7-fcbb800ec915-service-ca-bundle\") pod \"router-default-699696b566-qzknj\" (UID: \"9733272a-e52f-47ca-abc7-fcbb800ec915\") " pod="openshift-ingress/router-default-699696b566-qzknj" Sep 10 02:49:57.058526 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:57.058324 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9733272a-e52f-47ca-abc7-fcbb800ec915-metrics-certs\") pod \"router-default-699696b566-qzknj\" (UID: \"9733272a-e52f-47ca-abc7-fcbb800ec915\") " pod="openshift-ingress/router-default-699696b566-qzknj" Sep 10 02:49:57.058526 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:57.058522 2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 10 02:49:57.058796 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:57.058599 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9733272a-e52f-47ca-abc7-fcbb800ec915-metrics-certs podName:9733272a-e52f-47ca-abc7-fcbb800ec915 nodeName:}" failed. No retries permitted until 2026-09-10 02:50:05.058578979 +0000 UTC m=+63.905408248 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9733272a-e52f-47ca-abc7-fcbb800ec915-metrics-certs") pod "router-default-699696b566-qzknj" (UID: "9733272a-e52f-47ca-abc7-fcbb800ec915") : secret "router-metrics-certs-default" not found Sep 10 02:49:57.062521 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:57.059020 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9733272a-e52f-47ca-abc7-fcbb800ec915-service-ca-bundle podName:9733272a-e52f-47ca-abc7-fcbb800ec915 nodeName:}" failed. No retries permitted until 2026-09-10 02:50:05.058996317 +0000 UTC m=+63.905825585 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9733272a-e52f-47ca-abc7-fcbb800ec915-service-ca-bundle") pod "router-default-699696b566-qzknj" (UID: "9733272a-e52f-47ca-abc7-fcbb800ec915") : configmap references non-existent config key: service-ca.crt Sep 10 02:49:57.260053 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:57.259950 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/49552b19-52f2-41d8-bc59-40bb3a992d6a-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-6w544\" (UID: \"49552b19-52f2-41d8-bc59-40bb3a992d6a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-6w544" Sep 10 02:49:57.260053 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:57.260018 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4d65d823-e75e-4f53-b9f1-c6a5ed728133-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-7xqrx\" (UID: \"4d65d823-e75e-4f53-b9f1-c6a5ed728133\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xqrx" Sep 10 02:49:57.260289 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:57.260147 2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 02:49:57.260289 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:57.260178 2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 02:49:57.260289 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:57.260245 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49552b19-52f2-41d8-bc59-40bb3a992d6a-samples-operator-tls podName:49552b19-52f2-41d8-bc59-40bb3a992d6a nodeName:}" failed. No retries permitted until 2026-09-10 02:50:05.260222224 +0000 UTC m=+64.107051493 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/49552b19-52f2-41d8-bc59-40bb3a992d6a-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-6w544" (UID: "49552b19-52f2-41d8-bc59-40bb3a992d6a") : secret "samples-operator-tls" not found Sep 10 02:49:57.260289 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:57.260265 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d65d823-e75e-4f53-b9f1-c6a5ed728133-cluster-monitoring-operator-tls podName:4d65d823-e75e-4f53-b9f1-c6a5ed728133 nodeName:}" failed. No retries permitted until 2026-09-10 02:50:05.260255455 +0000 UTC m=+64.107084707 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4d65d823-e75e-4f53-b9f1-c6a5ed728133-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-7xqrx" (UID: "4d65d823-e75e-4f53-b9f1-c6a5ed728133") : secret "cluster-monitoring-operator-tls" not found Sep 10 02:49:57.917292 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:57.917206 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ghs46_a5210aa8-4171-4cc3-b65b-6237a9aee1d6/console-operator/1.log" Sep 10 02:49:57.918199 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:57.918179 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ghs46_a5210aa8-4171-4cc3-b65b-6237a9aee1d6/console-operator/0.log" Sep 10 02:49:57.918366 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:57.918350 2560 generic.go:356] "Generic (PLEG): container finished" podID="a5210aa8-4171-4cc3-b65b-6237a9aee1d6" containerID="1b3e3f5915398325cdfd5543d12df428b2b68d8eddb0953d8c68cfc2b8a3cada" exitCode=255 Sep 10 02:49:57.918611 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:57.918584 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-ghs46" event={"ID":"a5210aa8-4171-4cc3-b65b-6237a9aee1d6","Type":"ContainerDied","Data":"1b3e3f5915398325cdfd5543d12df428b2b68d8eddb0953d8c68cfc2b8a3cada"} Sep 10 02:49:57.918685 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:57.918660 2560 scope.go:117] "RemoveContainer" containerID="54d3b4804f8d845b6fef4cd75a4fbddd1e1add7847575444590ee0ec2ec0ff87" Sep 10 02:49:57.921268 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:57.921242 2560 scope.go:117] "RemoveContainer" containerID="1b3e3f5915398325cdfd5543d12df428b2b68d8eddb0953d8c68cfc2b8a3cada" Sep 10 02:49:57.921692 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:57.921431 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-ghs46_openshift-console-operator(a5210aa8-4171-4cc3-b65b-6237a9aee1d6)\"" pod="openshift-console-operator/console-operator-85b454874b-ghs46" podUID="a5210aa8-4171-4cc3-b65b-6237a9aee1d6" Sep 10 02:49:58.029746 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:58.029706 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2wt7g"] Sep 10 02:49:58.032663 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:58.032643 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2wt7g" Sep 10 02:49:58.038248 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:58.038221 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-k6lnt\"" Sep 10 02:49:58.038483 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:58.038458 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 10 02:49:58.038593 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:58.038533 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 10 02:49:58.066008 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:58.065973 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2wt7g"] Sep 10 02:49:58.068867 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:58.068731 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8502f598-0d8b-429b-ab5c-c23c284b2fdb-crio-socket\") pod \"insights-runtime-extractor-2wt7g\" (UID: \"8502f598-0d8b-429b-ab5c-c23c284b2fdb\") " pod="openshift-insights/insights-runtime-extractor-2wt7g" Sep 10 02:49:58.069014 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:58.068967 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8502f598-0d8b-429b-ab5c-c23c284b2fdb-data-volume\") pod \"insights-runtime-extractor-2wt7g\" (UID: \"8502f598-0d8b-429b-ab5c-c23c284b2fdb\") " pod="openshift-insights/insights-runtime-extractor-2wt7g" Sep 10 02:49:58.069089 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:58.069032 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kp7l\" (UniqueName: \"kubernetes.io/projected/8502f598-0d8b-429b-ab5c-c23c284b2fdb-kube-api-access-8kp7l\") pod \"insights-runtime-extractor-2wt7g\" (UID: \"8502f598-0d8b-429b-ab5c-c23c284b2fdb\") " pod="openshift-insights/insights-runtime-extractor-2wt7g" Sep 10 02:49:58.069089 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:58.069065 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8502f598-0d8b-429b-ab5c-c23c284b2fdb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2wt7g\" (UID: \"8502f598-0d8b-429b-ab5c-c23c284b2fdb\") " pod="openshift-insights/insights-runtime-extractor-2wt7g" Sep 10 02:49:58.069187 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:58.069099 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8502f598-0d8b-429b-ab5c-c23c284b2fdb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2wt7g\" (UID: \"8502f598-0d8b-429b-ab5c-c23c284b2fdb\") " pod="openshift-insights/insights-runtime-extractor-2wt7g" Sep 10 02:49:58.169588 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:58.169484 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8502f598-0d8b-429b-ab5c-c23c284b2fdb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2wt7g\" (UID: \"8502f598-0d8b-429b-ab5c-c23c284b2fdb\") " pod="openshift-insights/insights-runtime-extractor-2wt7g" Sep 10 02:49:58.169588 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:58.169554 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8502f598-0d8b-429b-ab5c-c23c284b2fdb-crio-socket\") pod \"insights-runtime-extractor-2wt7g\" (UID: \"8502f598-0d8b-429b-ab5c-c23c284b2fdb\") " pod="openshift-insights/insights-runtime-extractor-2wt7g" Sep 10 02:49:58.169811 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:58.169600 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 02:49:58.169811 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:58.169669 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8502f598-0d8b-429b-ab5c-c23c284b2fdb-kube-rbac-proxy-cm podName:8502f598-0d8b-429b-ab5c-c23c284b2fdb nodeName:}" failed. No retries permitted until 2026-09-10 02:49:58.669649912 +0000 UTC m=+57.516479177 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8502f598-0d8b-429b-ab5c-c23c284b2fdb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2wt7g" (UID: "8502f598-0d8b-429b-ab5c-c23c284b2fdb") : configmap "kube-rbac-proxy" not found Sep 10 02:49:58.169811 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:58.169729 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8502f598-0d8b-429b-ab5c-c23c284b2fdb-data-volume\") pod \"insights-runtime-extractor-2wt7g\" (UID: \"8502f598-0d8b-429b-ab5c-c23c284b2fdb\") " pod="openshift-insights/insights-runtime-extractor-2wt7g" Sep 10 02:49:58.169811 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:58.169765 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kp7l\" (UniqueName: \"kubernetes.io/projected/8502f598-0d8b-429b-ab5c-c23c284b2fdb-kube-api-access-8kp7l\") pod \"insights-runtime-extractor-2wt7g\" (UID: \"8502f598-0d8b-429b-ab5c-c23c284b2fdb\") " pod="openshift-insights/insights-runtime-extractor-2wt7g" Sep 10 02:49:58.169811 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:58.169778 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8502f598-0d8b-429b-ab5c-c23c284b2fdb-crio-socket\") pod \"insights-runtime-extractor-2wt7g\" (UID: \"8502f598-0d8b-429b-ab5c-c23c284b2fdb\") " pod="openshift-insights/insights-runtime-extractor-2wt7g" Sep 10 02:49:58.169811 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:58.169793 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8502f598-0d8b-429b-ab5c-c23c284b2fdb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2wt7g\" (UID: \"8502f598-0d8b-429b-ab5c-c23c284b2fdb\") " pod="openshift-insights/insights-runtime-extractor-2wt7g" Sep 10 02:49:58.170107 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:58.169868 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 02:49:58.170107 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:58.169916 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8502f598-0d8b-429b-ab5c-c23c284b2fdb-insights-runtime-extractor-tls podName:8502f598-0d8b-429b-ab5c-c23c284b2fdb nodeName:}" failed. No retries permitted until 2026-09-10 02:49:58.669903726 +0000 UTC m=+57.516732996 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8502f598-0d8b-429b-ab5c-c23c284b2fdb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2wt7g" (UID: "8502f598-0d8b-429b-ab5c-c23c284b2fdb") : secret "insights-runtime-extractor-tls" not found Sep 10 02:49:58.170107 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:58.170063 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8502f598-0d8b-429b-ab5c-c23c284b2fdb-data-volume\") pod \"insights-runtime-extractor-2wt7g\" (UID: \"8502f598-0d8b-429b-ab5c-c23c284b2fdb\") " pod="openshift-insights/insights-runtime-extractor-2wt7g" Sep 10 02:49:58.197658 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:58.197626 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kp7l\" (UniqueName: \"kubernetes.io/projected/8502f598-0d8b-429b-ab5c-c23c284b2fdb-kube-api-access-8kp7l\") pod \"insights-runtime-extractor-2wt7g\" (UID: \"8502f598-0d8b-429b-ab5c-c23c284b2fdb\") " pod="openshift-insights/insights-runtime-extractor-2wt7g" Sep 10 02:49:58.674722 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:58.674686 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8502f598-0d8b-429b-ab5c-c23c284b2fdb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2wt7g\" (UID: \"8502f598-0d8b-429b-ab5c-c23c284b2fdb\") " pod="openshift-insights/insights-runtime-extractor-2wt7g" Sep 10 02:49:58.674897 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:58.674813 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 02:49:58.674897 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:58.674844 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8502f598-0d8b-429b-ab5c-c23c284b2fdb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2wt7g\" (UID: \"8502f598-0d8b-429b-ab5c-c23c284b2fdb\") " pod="openshift-insights/insights-runtime-extractor-2wt7g" Sep 10 02:49:58.674897 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:58.674886 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8502f598-0d8b-429b-ab5c-c23c284b2fdb-kube-rbac-proxy-cm podName:8502f598-0d8b-429b-ab5c-c23c284b2fdb nodeName:}" failed. No retries permitted until 2026-09-10 02:49:59.674869561 +0000 UTC m=+58.521698824 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8502f598-0d8b-429b-ab5c-c23c284b2fdb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2wt7g" (UID: "8502f598-0d8b-429b-ab5c-c23c284b2fdb") : configmap "kube-rbac-proxy" not found Sep 10 02:49:58.675043 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:58.674944 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 02:49:58.675043 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:58.674994 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8502f598-0d8b-429b-ab5c-c23c284b2fdb-insights-runtime-extractor-tls podName:8502f598-0d8b-429b-ab5c-c23c284b2fdb nodeName:}" failed. No retries permitted until 2026-09-10 02:49:59.674977942 +0000 UTC m=+58.521807188 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8502f598-0d8b-429b-ab5c-c23c284b2fdb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2wt7g" (UID: "8502f598-0d8b-429b-ab5c-c23c284b2fdb") : secret "insights-runtime-extractor-tls" not found Sep 10 02:49:58.923768 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:58.923742 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ghs46_a5210aa8-4171-4cc3-b65b-6237a9aee1d6/console-operator/1.log" Sep 10 02:49:58.924133 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:58.924094 2560 scope.go:117] "RemoveContainer" containerID="1b3e3f5915398325cdfd5543d12df428b2b68d8eddb0953d8c68cfc2b8a3cada" Sep 10 02:49:58.924256 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:58.924238 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-ghs46_openshift-console-operator(a5210aa8-4171-4cc3-b65b-6237a9aee1d6)\"" pod="openshift-console-operator/console-operator-85b454874b-ghs46" podUID="a5210aa8-4171-4cc3-b65b-6237a9aee1d6" Sep 10 02:49:59.683719 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:59.683674 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8502f598-0d8b-429b-ab5c-c23c284b2fdb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2wt7g\" (UID: \"8502f598-0d8b-429b-ab5c-c23c284b2fdb\") " pod="openshift-insights/insights-runtime-extractor-2wt7g" Sep 10 02:49:59.683928 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:59.683796 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 02:49:59.683928 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:59.683862 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8502f598-0d8b-429b-ab5c-c23c284b2fdb-kube-rbac-proxy-cm podName:8502f598-0d8b-429b-ab5c-c23c284b2fdb nodeName:}" failed. No retries permitted until 2026-09-10 02:50:01.68384635 +0000 UTC m=+60.530675609 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8502f598-0d8b-429b-ab5c-c23c284b2fdb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2wt7g" (UID: "8502f598-0d8b-429b-ab5c-c23c284b2fdb") : configmap "kube-rbac-proxy" not found Sep 10 02:49:59.683928 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:59.683908 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8502f598-0d8b-429b-ab5c-c23c284b2fdb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2wt7g\" (UID: \"8502f598-0d8b-429b-ab5c-c23c284b2fdb\") " pod="openshift-insights/insights-runtime-extractor-2wt7g" Sep 10 02:49:59.684093 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:59.683995 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 02:49:59.684093 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:59.684026 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8502f598-0d8b-429b-ab5c-c23c284b2fdb-insights-runtime-extractor-tls podName:8502f598-0d8b-429b-ab5c-c23c284b2fdb nodeName:}" failed. No retries permitted until 2026-09-10 02:50:01.68401886 +0000 UTC m=+60.530848119 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8502f598-0d8b-429b-ab5c-c23c284b2fdb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2wt7g" (UID: "8502f598-0d8b-429b-ab5c-c23c284b2fdb") : secret "insights-runtime-extractor-tls" not found Sep 10 02:49:59.804007 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:59.803965 2560 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-ghs46" Sep 10 02:49:59.804342 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:59.804204 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-ghs46" Sep 10 02:49:59.930425 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:59.927976 2560 scope.go:117] "RemoveContainer" containerID="1b3e3f5915398325cdfd5543d12df428b2b68d8eddb0953d8c68cfc2b8a3cada" Sep 10 02:49:59.930425 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:49:59.928254 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-ghs46_openshift-console-operator(a5210aa8-4171-4cc3-b65b-6237a9aee1d6)\"" pod="openshift-console-operator/console-operator-85b454874b-ghs46" podUID="a5210aa8-4171-4cc3-b65b-6237a9aee1d6" Sep 10 02:49:59.942235 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:49:59.942156 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kkzdh_6fdc2579-93fb-494d-b179-46cf1db9aaa8/dns-node-resolver/0.log" Sep 10 02:50:01.135450 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:01.135421 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mnpps_936410ac-13a7-43ca-a51f-b0bd96649ff6/node-ca/0.log" Sep 10 02:50:01.701399 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:01.701355 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8502f598-0d8b-429b-ab5c-c23c284b2fdb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2wt7g\" (UID: \"8502f598-0d8b-429b-ab5c-c23c284b2fdb\") " pod="openshift-insights/insights-runtime-extractor-2wt7g" Sep 10 02:50:01.701399 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:01.701405 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8502f598-0d8b-429b-ab5c-c23c284b2fdb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2wt7g\" (UID: \"8502f598-0d8b-429b-ab5c-c23c284b2fdb\") " pod="openshift-insights/insights-runtime-extractor-2wt7g" Sep 10 02:50:01.701655 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:50:01.701524 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 02:50:01.701655 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:50:01.701562 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 02:50:01.701655 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:50:01.701607 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8502f598-0d8b-429b-ab5c-c23c284b2fdb-insights-runtime-extractor-tls podName:8502f598-0d8b-429b-ab5c-c23c284b2fdb nodeName:}" failed. No retries permitted until 2026-09-10 02:50:05.701589957 +0000 UTC m=+64.548419220 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8502f598-0d8b-429b-ab5c-c23c284b2fdb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2wt7g" (UID: "8502f598-0d8b-429b-ab5c-c23c284b2fdb") : secret "insights-runtime-extractor-tls" not found Sep 10 02:50:01.701655 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:50:01.701621 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8502f598-0d8b-429b-ab5c-c23c284b2fdb-kube-rbac-proxy-cm podName:8502f598-0d8b-429b-ab5c-c23c284b2fdb nodeName:}" failed. No retries permitted until 2026-09-10 02:50:05.701615139 +0000 UTC m=+64.548444385 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8502f598-0d8b-429b-ab5c-c23c284b2fdb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2wt7g" (UID: "8502f598-0d8b-429b-ab5c-c23c284b2fdb") : configmap "kube-rbac-proxy" not found Sep 10 02:50:01.815456 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:01.815430 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-swhbm" Sep 10 02:50:01.943175 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:01.943138 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mpnm6_007e1c44-e993-4808-b999-05799e398c2f/serve-healthcheck-canary/0.log" Sep 10 02:50:05.133047 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:05.133012 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9733272a-e52f-47ca-abc7-fcbb800ec915-service-ca-bundle\") pod \"router-default-699696b566-qzknj\" (UID: \"9733272a-e52f-47ca-abc7-fcbb800ec915\") " pod="openshift-ingress/router-default-699696b566-qzknj" Sep 10 02:50:05.133432 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:05.133076 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9733272a-e52f-47ca-abc7-fcbb800ec915-metrics-certs\") pod \"router-default-699696b566-qzknj\" (UID: \"9733272a-e52f-47ca-abc7-fcbb800ec915\") " pod="openshift-ingress/router-default-699696b566-qzknj" Sep 10 02:50:05.133432 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:50:05.133160 2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 10 02:50:05.133432 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:50:05.133231 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9733272a-e52f-47ca-abc7-fcbb800ec915-service-ca-bundle podName:9733272a-e52f-47ca-abc7-fcbb800ec915 nodeName:}" failed. No retries permitted until 2026-09-10 02:50:21.133208243 +0000 UTC m=+79.980037508 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9733272a-e52f-47ca-abc7-fcbb800ec915-service-ca-bundle") pod "router-default-699696b566-qzknj" (UID: "9733272a-e52f-47ca-abc7-fcbb800ec915") : configmap references non-existent config key: service-ca.crt Sep 10 02:50:05.133432 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:50:05.133258 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9733272a-e52f-47ca-abc7-fcbb800ec915-metrics-certs podName:9733272a-e52f-47ca-abc7-fcbb800ec915 nodeName:}" failed. No retries permitted until 2026-09-10 02:50:21.133250415 +0000 UTC m=+79.980079661 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9733272a-e52f-47ca-abc7-fcbb800ec915-metrics-certs") pod "router-default-699696b566-qzknj" (UID: "9733272a-e52f-47ca-abc7-fcbb800ec915") : secret "router-metrics-certs-default" not found Sep 10 02:50:05.335695 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:05.335664 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/49552b19-52f2-41d8-bc59-40bb3a992d6a-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-6w544\" (UID: \"49552b19-52f2-41d8-bc59-40bb3a992d6a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-6w544" Sep 10 02:50:05.335893 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:05.335705 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4d65d823-e75e-4f53-b9f1-c6a5ed728133-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-7xqrx\" (UID: \"4d65d823-e75e-4f53-b9f1-c6a5ed728133\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xqrx" Sep 10 02:50:05.335893 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:50:05.335848 2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 02:50:05.335893 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:50:05.335877 2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 02:50:05.336072 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:50:05.335925 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49552b19-52f2-41d8-bc59-40bb3a992d6a-samples-operator-tls podName:49552b19-52f2-41d8-bc59-40bb3a992d6a nodeName:}" failed. No retries permitted until 2026-09-10 02:50:21.335905826 +0000 UTC m=+80.182735093 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/49552b19-52f2-41d8-bc59-40bb3a992d6a-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-6w544" (UID: "49552b19-52f2-41d8-bc59-40bb3a992d6a") : secret "samples-operator-tls" not found Sep 10 02:50:05.336072 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:50:05.335944 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d65d823-e75e-4f53-b9f1-c6a5ed728133-cluster-monitoring-operator-tls podName:4d65d823-e75e-4f53-b9f1-c6a5ed728133 nodeName:}" failed. No retries permitted until 2026-09-10 02:50:21.335935331 +0000 UTC m=+80.182764577 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4d65d823-e75e-4f53-b9f1-c6a5ed728133-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-7xqrx" (UID: "4d65d823-e75e-4f53-b9f1-c6a5ed728133") : secret "cluster-monitoring-operator-tls" not found Sep 10 02:50:05.739628 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:05.739591 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8502f598-0d8b-429b-ab5c-c23c284b2fdb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2wt7g\" (UID: \"8502f598-0d8b-429b-ab5c-c23c284b2fdb\") " pod="openshift-insights/insights-runtime-extractor-2wt7g" Sep 10 02:50:05.739628 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:05.739630 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8502f598-0d8b-429b-ab5c-c23c284b2fdb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2wt7g\" (UID: \"8502f598-0d8b-429b-ab5c-c23c284b2fdb\") " pod="openshift-insights/insights-runtime-extractor-2wt7g" Sep 10 02:50:05.739861 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:50:05.739742 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 02:50:05.739861 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:50:05.739799 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 02:50:05.739861 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:50:05.739804 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8502f598-0d8b-429b-ab5c-c23c284b2fdb-insights-runtime-extractor-tls podName:8502f598-0d8b-429b-ab5c-c23c284b2fdb nodeName:}" failed. No retries permitted until 2026-09-10 02:50:13.739788075 +0000 UTC m=+72.586617339 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8502f598-0d8b-429b-ab5c-c23c284b2fdb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2wt7g" (UID: "8502f598-0d8b-429b-ab5c-c23c284b2fdb") : secret "insights-runtime-extractor-tls" not found Sep 10 02:50:05.739973 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:50:05.739870 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8502f598-0d8b-429b-ab5c-c23c284b2fdb-kube-rbac-proxy-cm podName:8502f598-0d8b-429b-ab5c-c23c284b2fdb nodeName:}" failed. No retries permitted until 2026-09-10 02:50:13.739852941 +0000 UTC m=+72.586682190 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8502f598-0d8b-429b-ab5c-c23c284b2fdb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2wt7g" (UID: "8502f598-0d8b-429b-ab5c-c23c284b2fdb") : configmap "kube-rbac-proxy" not found Sep 10 02:50:07.454933 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:07.454892 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ff1e29e-7033-490c-9316-def0076b8f7e-original-pull-secret\") pod \"global-pull-secret-syncer-sdxng\" (UID: \"9ff1e29e-7033-490c-9316-def0076b8f7e\") " pod="kube-system/global-pull-secret-syncer-sdxng" Sep 10 02:50:07.455417 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:07.454951 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e-metrics-certs\") pod \"network-metrics-daemon-d2sbz\" (UID: \"c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e\") " pod="openshift-multus/network-metrics-daemon-d2sbz" Sep 10 02:50:07.457821 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:07.457800 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 02:50:07.457951 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:07.457938 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 10 02:50:07.465179 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:50:07.465160 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 10 02:50:07.465234 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:50:07.465224 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e-metrics-certs podName:c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e nodeName:}" failed. No retries permitted until 2026-09-10 02:51:11.465204687 +0000 UTC m=+130.312033932 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e-metrics-certs") pod "network-metrics-daemon-d2sbz" (UID: "c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e") : secret "metrics-daemon-secret" not found Sep 10 02:50:07.467451 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:07.467431 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ff1e29e-7033-490c-9316-def0076b8f7e-original-pull-secret\") pod \"global-pull-secret-syncer-sdxng\" (UID: \"9ff1e29e-7033-490c-9316-def0076b8f7e\") " pod="kube-system/global-pull-secret-syncer-sdxng" Sep 10 02:50:07.555873 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:07.555833 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt95b\" (UniqueName: \"kubernetes.io/projected/690238d6-9e65-42b8-b042-d1c3efbc1e45-kube-api-access-pt95b\") pod \"network-check-target-fqfck\" (UID: \"690238d6-9e65-42b8-b042-d1c3efbc1e45\") " pod="openshift-network-diagnostics/network-check-target-fqfck" Sep 10 02:50:07.556055 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:07.555909 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sdxng" Sep 10 02:50:07.558445 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:07.558420 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt95b\" (UniqueName: \"kubernetes.io/projected/690238d6-9e65-42b8-b042-d1c3efbc1e45-kube-api-access-pt95b\") pod \"network-check-target-fqfck\" (UID: \"690238d6-9e65-42b8-b042-d1c3efbc1e45\") " pod="openshift-network-diagnostics/network-check-target-fqfck" Sep 10 02:50:07.563640 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:07.563625 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lc7sq\"" Sep 10 02:50:07.571230 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:07.571203 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fqfck" Sep 10 02:50:07.827999 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:07.827918 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sdxng"] Sep 10 02:50:07.832186 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:50:07.832154 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ff1e29e_7033_490c_9316_def0076b8f7e.slice/crio-70d4359fc7559c0d8e0f65d2cc3d93ffa7257052c2eb1101d338f28128ad58f6 WatchSource:0}: Error finding container 70d4359fc7559c0d8e0f65d2cc3d93ffa7257052c2eb1101d338f28128ad58f6: Status 404 returned error can't find the container with id 70d4359fc7559c0d8e0f65d2cc3d93ffa7257052c2eb1101d338f28128ad58f6 Sep 10 02:50:07.848753 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:07.848726 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fqfck"] Sep 10 02:50:07.851492 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:50:07.851469 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod690238d6_9e65_42b8_b042_d1c3efbc1e45.slice/crio-6ee86eae1449f274d8834f02cb65b7edf0252a945cd992da40a5bba9015ba944 WatchSource:0}: Error finding container 6ee86eae1449f274d8834f02cb65b7edf0252a945cd992da40a5bba9015ba944: Status 404 returned error can't find the container with id 6ee86eae1449f274d8834f02cb65b7edf0252a945cd992da40a5bba9015ba944 Sep 10 02:50:07.953526 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:07.953484 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fqfck" event={"ID":"690238d6-9e65-42b8-b042-d1c3efbc1e45","Type":"ContainerStarted","Data":"fe63dee8be13347a7f93c62bd3a00889d03ff246028604fd5f0bc90f528bc6dd"} Sep 10 02:50:07.953725 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:07.953553 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fqfck" event={"ID":"690238d6-9e65-42b8-b042-d1c3efbc1e45","Type":"ContainerStarted","Data":"6ee86eae1449f274d8834f02cb65b7edf0252a945cd992da40a5bba9015ba944"} Sep 10 02:50:07.953725 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:07.953597 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fqfck" Sep 10 02:50:07.954762 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:07.954731 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sdxng" event={"ID":"9ff1e29e-7033-490c-9316-def0076b8f7e","Type":"ContainerStarted","Data":"70d4359fc7559c0d8e0f65d2cc3d93ffa7257052c2eb1101d338f28128ad58f6"} Sep 10 02:50:07.958867 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:07.958843 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/98643336-46ce-4f17-acb2-d8110a5bfb84-metrics-tls\") pod \"dns-default-2s6nr\" (UID: \"98643336-46ce-4f17-acb2-d8110a5bfb84\") " pod="openshift-dns/dns-default-2s6nr" Sep 10 02:50:07.958945 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:07.958893 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-registry-tls\") pod \"image-registry-75f8b47547-m9np9\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " pod="openshift-image-registry/image-registry-75f8b47547-m9np9" Sep 10 02:50:07.958998 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:50:07.958987 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 02:50:07.959036 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:50:07.959001 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75f8b47547-m9np9: secret "image-registry-tls" not found Sep 10 02:50:07.959071 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:50:07.959045 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-registry-tls podName:e4d56e77-0fdc-4733-b06f-2451d59f8343 nodeName:}" failed. No retries permitted until 2026-09-10 02:50:39.959032916 +0000 UTC m=+98.805862163 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-registry-tls") pod "image-registry-75f8b47547-m9np9" (UID: "e4d56e77-0fdc-4733-b06f-2451d59f8343") : secret "image-registry-tls" not found Sep 10 02:50:07.959071 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:50:07.958986 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 02:50:07.959141 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:50:07.959119 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98643336-46ce-4f17-acb2-d8110a5bfb84-metrics-tls podName:98643336-46ce-4f17-acb2-d8110a5bfb84 nodeName:}" failed. No retries permitted until 2026-09-10 02:50:39.959103826 +0000 UTC m=+98.805933075 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/98643336-46ce-4f17-acb2-d8110a5bfb84-metrics-tls") pod "dns-default-2s6nr" (UID: "98643336-46ce-4f17-acb2-d8110a5bfb84") : secret "dns-default-metrics-tls" not found Sep 10 02:50:07.997630 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:07.997571 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-fqfck" podStartSLOduration=65.99755113 podStartE2EDuration="1m5.99755113s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 02:49:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:50:07.997278527 +0000 UTC m=+66.844107796" watchObservedRunningTime="2026-09-10 02:50:07.99755113 +0000 UTC m=+66.844380399" Sep 10 02:50:10.640844 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:10.640812 2560 scope.go:117] "RemoveContainer" containerID="1b3e3f5915398325cdfd5543d12df428b2b68d8eddb0953d8c68cfc2b8a3cada" Sep 10 02:50:10.966971 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:10.966905 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ghs46_a5210aa8-4171-4cc3-b65b-6237a9aee1d6/console-operator/2.log" Sep 10 02:50:10.967321 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:10.967306 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ghs46_a5210aa8-4171-4cc3-b65b-6237a9aee1d6/console-operator/1.log" Sep 10 02:50:10.967367 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:10.967342 2560 generic.go:356] "Generic (PLEG): container finished" podID="a5210aa8-4171-4cc3-b65b-6237a9aee1d6" containerID="d4d59eaefd2fa9099835655195495866d3cb7731eba2bbcf9c16b0434ecfaff2" exitCode=255 Sep 10 02:50:10.967404 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:10.967387 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-ghs46" event={"ID":"a5210aa8-4171-4cc3-b65b-6237a9aee1d6","Type":"ContainerDied","Data":"d4d59eaefd2fa9099835655195495866d3cb7731eba2bbcf9c16b0434ecfaff2"} Sep 10 02:50:10.967444 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:10.967434 2560 scope.go:117] "RemoveContainer" containerID="1b3e3f5915398325cdfd5543d12df428b2b68d8eddb0953d8c68cfc2b8a3cada" Sep 10 02:50:10.967862 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:10.967836 2560 scope.go:117] "RemoveContainer" containerID="d4d59eaefd2fa9099835655195495866d3cb7731eba2bbcf9c16b0434ecfaff2" Sep 10 02:50:10.968085 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:50:10.968061 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-ghs46_openshift-console-operator(a5210aa8-4171-4cc3-b65b-6237a9aee1d6)\"" pod="openshift-console-operator/console-operator-85b454874b-ghs46" podUID="a5210aa8-4171-4cc3-b65b-6237a9aee1d6" Sep 10 02:50:12.974974 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:12.974934 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sdxng" event={"ID":"9ff1e29e-7033-490c-9316-def0076b8f7e","Type":"ContainerStarted","Data":"193cdbe6dab827faab21ac8ab0f32126a6f569ae1b6bcc3aedb0d36cff7aeae0"} Sep 10 02:50:12.976396 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:12.976373 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ghs46_a5210aa8-4171-4cc3-b65b-6237a9aee1d6/console-operator/2.log" Sep 10 02:50:12.991657 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:12.991609 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-sdxng" podStartSLOduration=66.457113466 podStartE2EDuration="1m10.991596313s" totalImagesPullingTime="4.534482847s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 02:49:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:50:12.991230177 +0000 UTC m=+71.838059444" watchObservedRunningTime="2026-09-10 02:50:12.991596313 +0000 UTC m=+71.838425611" Sep 10 02:50:13.812382 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:13.812343 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8502f598-0d8b-429b-ab5c-c23c284b2fdb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2wt7g\" (UID: \"8502f598-0d8b-429b-ab5c-c23c284b2fdb\") " pod="openshift-insights/insights-runtime-extractor-2wt7g" Sep 10 02:50:13.812382 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:13.812383 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8502f598-0d8b-429b-ab5c-c23c284b2fdb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2wt7g\" (UID: \"8502f598-0d8b-429b-ab5c-c23c284b2fdb\") " pod="openshift-insights/insights-runtime-extractor-2wt7g" Sep 10 02:50:13.812632 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:50:13.812532 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 02:50:13.812632 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:50:13.812604 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8502f598-0d8b-429b-ab5c-c23c284b2fdb-insights-runtime-extractor-tls podName:8502f598-0d8b-429b-ab5c-c23c284b2fdb nodeName:}" failed. No retries permitted until 2026-09-10 02:50:29.812586776 +0000 UTC m=+88.659416027 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8502f598-0d8b-429b-ab5c-c23c284b2fdb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2wt7g" (UID: "8502f598-0d8b-429b-ab5c-c23c284b2fdb") : secret "insights-runtime-extractor-tls" not found Sep 10 02:50:13.812982 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:13.812961 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8502f598-0d8b-429b-ab5c-c23c284b2fdb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2wt7g\" (UID: \"8502f598-0d8b-429b-ab5c-c23c284b2fdb\") " pod="openshift-insights/insights-runtime-extractor-2wt7g" Sep 10 02:50:19.804018 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:19.803980 2560 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-ghs46" Sep 10 02:50:19.804647 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:19.804056 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-ghs46" Sep 10 02:50:19.804647 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:19.804625 2560 scope.go:117] "RemoveContainer" containerID="d4d59eaefd2fa9099835655195495866d3cb7731eba2bbcf9c16b0434ecfaff2" Sep 10 02:50:19.804899 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:50:19.804858 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-ghs46_openshift-console-operator(a5210aa8-4171-4cc3-b65b-6237a9aee1d6)\"" pod="openshift-console-operator/console-operator-85b454874b-ghs46" podUID="a5210aa8-4171-4cc3-b65b-6237a9aee1d6" Sep 10 02:50:19.998809 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:19.998779 2560 scope.go:117] "RemoveContainer" containerID="d4d59eaefd2fa9099835655195495866d3cb7731eba2bbcf9c16b0434ecfaff2" Sep 10 02:50:19.998970 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:50:19.998939 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-ghs46_openshift-console-operator(a5210aa8-4171-4cc3-b65b-6237a9aee1d6)\"" pod="openshift-console-operator/console-operator-85b454874b-ghs46" podUID="a5210aa8-4171-4cc3-b65b-6237a9aee1d6" Sep 10 02:50:21.176598 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:21.176559 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9733272a-e52f-47ca-abc7-fcbb800ec915-service-ca-bundle\") pod \"router-default-699696b566-qzknj\" (UID: \"9733272a-e52f-47ca-abc7-fcbb800ec915\") " pod="openshift-ingress/router-default-699696b566-qzknj" Sep 10 02:50:21.176967 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:21.176621 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9733272a-e52f-47ca-abc7-fcbb800ec915-metrics-certs\") pod \"router-default-699696b566-qzknj\" (UID: \"9733272a-e52f-47ca-abc7-fcbb800ec915\") " pod="openshift-ingress/router-default-699696b566-qzknj" Sep 10 02:50:21.177194 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:21.177171 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9733272a-e52f-47ca-abc7-fcbb800ec915-service-ca-bundle\") pod \"router-default-699696b566-qzknj\" (UID: \"9733272a-e52f-47ca-abc7-fcbb800ec915\") " pod="openshift-ingress/router-default-699696b566-qzknj" Sep 10 02:50:21.179085 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:21.179064 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9733272a-e52f-47ca-abc7-fcbb800ec915-metrics-certs\") pod \"router-default-699696b566-qzknj\" (UID: \"9733272a-e52f-47ca-abc7-fcbb800ec915\") " pod="openshift-ingress/router-default-699696b566-qzknj" Sep 10 02:50:21.378794 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:21.378754 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/49552b19-52f2-41d8-bc59-40bb3a992d6a-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-6w544\" (UID: \"49552b19-52f2-41d8-bc59-40bb3a992d6a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-6w544" Sep 10 02:50:21.378794 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:21.378803 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4d65d823-e75e-4f53-b9f1-c6a5ed728133-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-7xqrx\" (UID: \"4d65d823-e75e-4f53-b9f1-c6a5ed728133\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xqrx" Sep 10 02:50:21.381386 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:21.381356 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4d65d823-e75e-4f53-b9f1-c6a5ed728133-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-7xqrx\" (UID: \"4d65d823-e75e-4f53-b9f1-c6a5ed728133\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xqrx" Sep 10 02:50:21.381534 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:21.381429 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/49552b19-52f2-41d8-bc59-40bb3a992d6a-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-6w544\" (UID: \"49552b19-52f2-41d8-bc59-40bb3a992d6a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-6w544" Sep 10 02:50:21.406051 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:21.406022 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-w6765\"" Sep 10 02:50:21.414246 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:21.414224 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-699696b566-qzknj" Sep 10 02:50:21.536151 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:21.536124 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-xvq4l\"" Sep 10 02:50:21.544157 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:21.544123 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xqrx" Sep 10 02:50:21.548813 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:21.548785 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-688qk\"" Sep 10 02:50:21.556860 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:21.556836 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-6w544" Sep 10 02:50:21.689433 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:21.689292 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-699696b566-qzknj"] Sep 10 02:50:21.692283 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:50:21.692259 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9733272a_e52f_47ca_abc7_fcbb800ec915.slice/crio-b907357ae3d5daff2f7f6061c60668c502c7709916fea376568ffec18ef169d1 WatchSource:0}: Error finding container b907357ae3d5daff2f7f6061c60668c502c7709916fea376568ffec18ef169d1: Status 404 returned error can't find the container with id b907357ae3d5daff2f7f6061c60668c502c7709916fea376568ffec18ef169d1 Sep 10 02:50:21.841320 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:21.841274 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xqrx"] Sep 10 02:50:21.857368 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:50:21.857339 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d65d823_e75e_4f53_b9f1_c6a5ed728133.slice/crio-f7fb19803ccf7aca9cfbe31f8725dbb298cfb72b1f0ed672d4d75049a38feaaa WatchSource:0}: Error finding container f7fb19803ccf7aca9cfbe31f8725dbb298cfb72b1f0ed672d4d75049a38feaaa: Status 404 returned error can't find the container with id f7fb19803ccf7aca9cfbe31f8725dbb298cfb72b1f0ed672d4d75049a38feaaa Sep 10 02:50:21.873430 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:21.873395 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-6w544"] Sep 10 02:50:22.007194 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:22.007111 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-6w544" event={"ID":"49552b19-52f2-41d8-bc59-40bb3a992d6a","Type":"ContainerStarted","Data":"9c8caff331bcd96c4d24812e30c6e3893d711ba110080473e997e01c768650b5"} Sep 10 02:50:22.008303 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:22.008272 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xqrx" event={"ID":"4d65d823-e75e-4f53-b9f1-c6a5ed728133","Type":"ContainerStarted","Data":"f7fb19803ccf7aca9cfbe31f8725dbb298cfb72b1f0ed672d4d75049a38feaaa"} Sep 10 02:50:22.009744 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:22.009721 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-699696b566-qzknj" event={"ID":"9733272a-e52f-47ca-abc7-fcbb800ec915","Type":"ContainerStarted","Data":"a0f62a093917adb51477719740de45927f4813ce9eb29472059fbea803823d78"} Sep 10 02:50:22.009830 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:22.009754 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-699696b566-qzknj" event={"ID":"9733272a-e52f-47ca-abc7-fcbb800ec915","Type":"ContainerStarted","Data":"b907357ae3d5daff2f7f6061c60668c502c7709916fea376568ffec18ef169d1"} Sep 10 02:50:22.129393 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:22.129344 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-699696b566-qzknj" podStartSLOduration=33.12932591 podStartE2EDuration="33.12932591s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 02:49:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:50:22.128642686 +0000 UTC m=+80.975471954" watchObservedRunningTime="2026-09-10 02:50:22.12932591 +0000 UTC m=+80.976155178" Sep 10 02:50:22.415421 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:22.415383 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-699696b566-qzknj" Sep 10 02:50:22.418171 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:22.418150 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-699696b566-qzknj" Sep 10 02:50:23.019762 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:23.019733 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-699696b566-qzknj" Sep 10 02:50:23.021184 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:23.021160 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-699696b566-qzknj" Sep 10 02:50:24.678900 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:24.678866 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-6ns6j"] Sep 10 02:50:24.682043 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:24.682026 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-6ns6j" Sep 10 02:50:24.684784 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:24.684768 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Sep 10 02:50:24.684858 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:24.684839 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-txhqq\"" Sep 10 02:50:24.690984 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:24.690963 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-6ns6j"] Sep 10 02:50:24.809338 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:24.809306 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8bfcfbe2-aea8-4f6e-85cc-ad0f431adfaf-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-6ns6j\" (UID: \"8bfcfbe2-aea8-4f6e-85cc-ad0f431adfaf\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-6ns6j" Sep 10 02:50:24.910673 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:24.910637 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8bfcfbe2-aea8-4f6e-85cc-ad0f431adfaf-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-6ns6j\" (UID: \"8bfcfbe2-aea8-4f6e-85cc-ad0f431adfaf\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-6ns6j" Sep 10 02:50:24.913384 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:24.913360 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8bfcfbe2-aea8-4f6e-85cc-ad0f431adfaf-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-6ns6j\" (UID: \"8bfcfbe2-aea8-4f6e-85cc-ad0f431adfaf\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-6ns6j" Sep 10 02:50:24.990995 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:24.990894 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-6ns6j" Sep 10 02:50:25.029421 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:25.029376 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-6w544" event={"ID":"49552b19-52f2-41d8-bc59-40bb3a992d6a","Type":"ContainerStarted","Data":"977e1c3df613d8a70ce46141e59831a979f1eb933100f3e85fdb1c5626eb75ec"} Sep 10 02:50:25.029622 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:25.029447 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-6w544" event={"ID":"49552b19-52f2-41d8-bc59-40bb3a992d6a","Type":"ContainerStarted","Data":"6e3d9a1b9c820a832837a2762c4272b8d032cf6eacefc8bdc415993012ceb072"} Sep 10 02:50:25.030906 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:25.030881 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xqrx" event={"ID":"4d65d823-e75e-4f53-b9f1-c6a5ed728133","Type":"ContainerStarted","Data":"3706181f2348fbcb5c0e5a860224af75c31487777c6c41553a4e4e3106cbfd6b"} Sep 10 02:50:25.087749 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:25.087692 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-6w544" podStartSLOduration=33.895715287 podStartE2EDuration="36.087673484s" totalImagesPullingTime="2.191958197s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 02:49:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:50:25.08677469 +0000 UTC m=+83.933603957" watchObservedRunningTime="2026-09-10 02:50:25.087673484 +0000 UTC m=+83.934502751" Sep 10 02:50:25.114354 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:25.114307 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-7xqrx" podStartSLOduration=33.862853401 podStartE2EDuration="36.11429204s" totalImagesPullingTime="2.251438639s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 02:49:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:50:25.113352001 +0000 UTC m=+83.960181269" watchObservedRunningTime="2026-09-10 02:50:25.11429204 +0000 UTC m=+83.961121306" Sep 10 02:50:25.260212 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:25.259887 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-6ns6j"] Sep 10 02:50:25.262424 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:50:25.262394 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bfcfbe2_aea8_4f6e_85cc_ad0f431adfaf.slice/crio-cc252c2a657d7939cc1f294e6c41b812c9965f6ab09f978e0bc590e2a17a80f9 WatchSource:0}: Error finding container cc252c2a657d7939cc1f294e6c41b812c9965f6ab09f978e0bc590e2a17a80f9: Status 404 returned error can't find the container with id cc252c2a657d7939cc1f294e6c41b812c9965f6ab09f978e0bc590e2a17a80f9 Sep 10 02:50:26.035773 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:26.035734 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-6ns6j" event={"ID":"8bfcfbe2-aea8-4f6e-85cc-ad0f431adfaf","Type":"ContainerStarted","Data":"cc252c2a657d7939cc1f294e6c41b812c9965f6ab09f978e0bc590e2a17a80f9"} Sep 10 02:50:27.042557 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:27.042443 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-6ns6j" event={"ID":"8bfcfbe2-aea8-4f6e-85cc-ad0f431adfaf","Type":"ContainerStarted","Data":"978cb8686cefd8bc4e84972d2989a30f90cc527d6fd354116a3025d210879045"} Sep 10 02:50:27.042994 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:27.042897 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-6ns6j" Sep 10 02:50:27.047768 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:27.047745 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-6ns6j" Sep 10 02:50:27.060663 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:27.060620 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-6ns6j" podStartSLOduration=1.531098605 podStartE2EDuration="3.060606894s" totalImagesPullingTime="1.529508289s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 02:50:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:50:27.060380414 +0000 UTC m=+85.907209683" watchObservedRunningTime="2026-09-10 02:50:27.060606894 +0000 UTC m=+85.907436161" Sep 10 02:50:29.854305 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:29.854273 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8502f598-0d8b-429b-ab5c-c23c284b2fdb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2wt7g\" (UID: \"8502f598-0d8b-429b-ab5c-c23c284b2fdb\") " pod="openshift-insights/insights-runtime-extractor-2wt7g" Sep 10 02:50:29.856658 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:29.856635 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8502f598-0d8b-429b-ab5c-c23c284b2fdb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2wt7g\" (UID: \"8502f598-0d8b-429b-ab5c-c23c284b2fdb\") " pod="openshift-insights/insights-runtime-extractor-2wt7g" Sep 10 02:50:30.148070 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:30.147991 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-k6lnt\"" Sep 10 02:50:30.155576 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:30.155556 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2wt7g" Sep 10 02:50:30.458800 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:30.458776 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2wt7g"] Sep 10 02:50:30.461158 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:50:30.461133 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8502f598_0d8b_429b_ab5c_c23c284b2fdb.slice/crio-c72f590e146ee62695865468a766ab846687d97be04d991e0403b4f0f76b8fe6 WatchSource:0}: Error finding container c72f590e146ee62695865468a766ab846687d97be04d991e0403b4f0f76b8fe6: Status 404 returned error can't find the container with id c72f590e146ee62695865468a766ab846687d97be04d991e0403b4f0f76b8fe6 Sep 10 02:50:31.057401 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:31.057365 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2wt7g" event={"ID":"8502f598-0d8b-429b-ab5c-c23c284b2fdb","Type":"ContainerStarted","Data":"798a67555565d00e5d85be0e68c83fef6b69f849486a92bedd3c645a8a00ab64"} Sep 10 02:50:31.057815 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:31.057441 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2wt7g" event={"ID":"8502f598-0d8b-429b-ab5c-c23c284b2fdb","Type":"ContainerStarted","Data":"c72f590e146ee62695865468a766ab846687d97be04d991e0403b4f0f76b8fe6"} Sep 10 02:50:32.062866 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:32.062780 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2wt7g" event={"ID":"8502f598-0d8b-429b-ab5c-c23c284b2fdb","Type":"ContainerStarted","Data":"438932f49fa6644aa31a6592147bc0a756bd062c1f3d58663c81fba95b1e364d"} Sep 10 02:50:33.640429 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:33.640396 2560 scope.go:117] "RemoveContainer" containerID="d4d59eaefd2fa9099835655195495866d3cb7731eba2bbcf9c16b0434ecfaff2" Sep 10 02:50:34.074857 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:34.074764 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2wt7g" event={"ID":"8502f598-0d8b-429b-ab5c-c23c284b2fdb","Type":"ContainerStarted","Data":"753993dbe2dca890d3c6ec987f4589c012957bf55770c69e0d52829c758c344b"} Sep 10 02:50:34.076780 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:34.076757 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ghs46_a5210aa8-4171-4cc3-b65b-6237a9aee1d6/console-operator/2.log" Sep 10 02:50:34.076907 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:34.076860 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-ghs46" event={"ID":"a5210aa8-4171-4cc3-b65b-6237a9aee1d6","Type":"ContainerStarted","Data":"b55328306afdce0d19d45d7c966a57acb5bf085338d5d0d3809397c6d1a1a3a8"} Sep 10 02:50:34.077152 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:34.077127 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-ghs46" Sep 10 02:50:34.102089 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:34.102039 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2wt7g" podStartSLOduration=34.59483253 podStartE2EDuration="37.102024139s" totalImagesPullingTime="2.507191609s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 02:49:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:50:34.101094541 +0000 UTC m=+92.947923832" watchObservedRunningTime="2026-09-10 02:50:34.102024139 +0000 UTC m=+92.948853406" Sep 10 02:50:34.122338 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:34.122285 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-ghs46" podStartSLOduration=38.646985566 podStartE2EDuration="45.122269261s" totalImagesPullingTime="6.475283695s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 02:49:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:50:34.12083125 +0000 UTC m=+92.967660530" watchObservedRunningTime="2026-09-10 02:50:34.122269261 +0000 UTC m=+92.969098566" Sep 10 02:50:34.692503 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:34.692473 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-ghs46" Sep 10 02:50:34.934635 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:34.934598 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-nbnc9"] Sep 10 02:50:34.936853 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:34.936838 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-nbnc9" Sep 10 02:50:34.945026 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:34.945003 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 10 02:50:34.945635 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:34.945618 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-wntht\"" Sep 10 02:50:34.945721 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:34.945705 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 10 02:50:35.028565 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.028533 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-nbnc9"] Sep 10 02:50:35.100352 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.100319 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gftbn\" (UniqueName: \"kubernetes.io/projected/c5d95d1c-394b-4750-99c5-ff8ea4afefd1-kube-api-access-gftbn\") pod \"downloads-85d98c998f-nbnc9\" (UID: \"c5d95d1c-394b-4750-99c5-ff8ea4afefd1\") " pod="openshift-console/downloads-85d98c998f-nbnc9" Sep 10 02:50:35.201387 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.201297 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gftbn\" (UniqueName: \"kubernetes.io/projected/c5d95d1c-394b-4750-99c5-ff8ea4afefd1-kube-api-access-gftbn\") pod \"downloads-85d98c998f-nbnc9\" (UID: \"c5d95d1c-394b-4750-99c5-ff8ea4afefd1\") " pod="openshift-console/downloads-85d98c998f-nbnc9" Sep 10 02:50:35.223646 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.223614 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gftbn\" (UniqueName: \"kubernetes.io/projected/c5d95d1c-394b-4750-99c5-ff8ea4afefd1-kube-api-access-gftbn\") pod \"downloads-85d98c998f-nbnc9\" (UID: \"c5d95d1c-394b-4750-99c5-ff8ea4afefd1\") " pod="openshift-console/downloads-85d98c998f-nbnc9" Sep 10 02:50:35.245524 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.245472 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-nbnc9" Sep 10 02:50:35.492810 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.492734 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-dnst6"] Sep 10 02:50:35.498935 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.498908 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dnst6" Sep 10 02:50:35.501281 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.501254 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 10 02:50:35.501555 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.501538 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 10 02:50:35.502312 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.502294 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xskv2\"" Sep 10 02:50:35.502451 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.502436 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 10 02:50:35.502986 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.502971 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 10 02:50:35.532669 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.532643 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-nbnc9"] Sep 10 02:50:35.535853 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:50:35.535823 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5d95d1c_394b_4750_99c5_ff8ea4afefd1.slice/crio-e0f7ebb6f9ff552c2241c43826c034ccdffe8e59f78728fcddf14fe2c2f2a8ca WatchSource:0}: Error finding container e0f7ebb6f9ff552c2241c43826c034ccdffe8e59f78728fcddf14fe2c2f2a8ca: Status 404 returned error can't find the container with id e0f7ebb6f9ff552c2241c43826c034ccdffe8e59f78728fcddf14fe2c2f2a8ca Sep 10 02:50:35.604732 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.604694 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/478b59a6-7c53-46ed-9a1d-944981dd4b7c-node-exporter-wtmp\") pod \"node-exporter-dnst6\" (UID: \"478b59a6-7c53-46ed-9a1d-944981dd4b7c\") " pod="openshift-monitoring/node-exporter-dnst6" Sep 10 02:50:35.604732 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.604734 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/478b59a6-7c53-46ed-9a1d-944981dd4b7c-metrics-client-ca\") pod \"node-exporter-dnst6\" (UID: \"478b59a6-7c53-46ed-9a1d-944981dd4b7c\") " pod="openshift-monitoring/node-exporter-dnst6" Sep 10 02:50:35.604981 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.604765 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5x5f8\" (UniqueName: \"kubernetes.io/projected/478b59a6-7c53-46ed-9a1d-944981dd4b7c-kube-api-access-5x5f8\") pod \"node-exporter-dnst6\" (UID: \"478b59a6-7c53-46ed-9a1d-944981dd4b7c\") " pod="openshift-monitoring/node-exporter-dnst6" Sep 10 02:50:35.604981 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.604857 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/478b59a6-7c53-46ed-9a1d-944981dd4b7c-root\") pod \"node-exporter-dnst6\" (UID: \"478b59a6-7c53-46ed-9a1d-944981dd4b7c\") " pod="openshift-monitoring/node-exporter-dnst6" Sep 10 02:50:35.604981 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.604925 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/478b59a6-7c53-46ed-9a1d-944981dd4b7c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dnst6\" (UID: \"478b59a6-7c53-46ed-9a1d-944981dd4b7c\") " pod="openshift-monitoring/node-exporter-dnst6" Sep 10 02:50:35.604981 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.604963 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/478b59a6-7c53-46ed-9a1d-944981dd4b7c-sys\") pod \"node-exporter-dnst6\" (UID: \"478b59a6-7c53-46ed-9a1d-944981dd4b7c\") " pod="openshift-monitoring/node-exporter-dnst6" Sep 10 02:50:35.605128 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.604999 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/478b59a6-7c53-46ed-9a1d-944981dd4b7c-node-exporter-textfile\") pod \"node-exporter-dnst6\" (UID: \"478b59a6-7c53-46ed-9a1d-944981dd4b7c\") " pod="openshift-monitoring/node-exporter-dnst6" Sep 10 02:50:35.605128 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.605037 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/478b59a6-7c53-46ed-9a1d-944981dd4b7c-node-exporter-accelerators-collector-config\") pod \"node-exporter-dnst6\" (UID: \"478b59a6-7c53-46ed-9a1d-944981dd4b7c\") " pod="openshift-monitoring/node-exporter-dnst6" Sep 10 02:50:35.605128 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.605086 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/478b59a6-7c53-46ed-9a1d-944981dd4b7c-node-exporter-tls\") pod \"node-exporter-dnst6\" (UID: \"478b59a6-7c53-46ed-9a1d-944981dd4b7c\") " pod="openshift-monitoring/node-exporter-dnst6" Sep 10 02:50:35.706006 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.705971 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/478b59a6-7c53-46ed-9a1d-944981dd4b7c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dnst6\" (UID: \"478b59a6-7c53-46ed-9a1d-944981dd4b7c\") " pod="openshift-monitoring/node-exporter-dnst6" Sep 10 02:50:35.706006 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.706011 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/478b59a6-7c53-46ed-9a1d-944981dd4b7c-sys\") pod \"node-exporter-dnst6\" (UID: \"478b59a6-7c53-46ed-9a1d-944981dd4b7c\") " pod="openshift-monitoring/node-exporter-dnst6" Sep 10 02:50:35.706491 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.706033 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/478b59a6-7c53-46ed-9a1d-944981dd4b7c-node-exporter-textfile\") pod \"node-exporter-dnst6\" (UID: \"478b59a6-7c53-46ed-9a1d-944981dd4b7c\") " pod="openshift-monitoring/node-exporter-dnst6" Sep 10 02:50:35.706491 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.706080 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/478b59a6-7c53-46ed-9a1d-944981dd4b7c-sys\") pod \"node-exporter-dnst6\" (UID: \"478b59a6-7c53-46ed-9a1d-944981dd4b7c\") " pod="openshift-monitoring/node-exporter-dnst6" Sep 10 02:50:35.706491 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.706110 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/478b59a6-7c53-46ed-9a1d-944981dd4b7c-node-exporter-accelerators-collector-config\") pod \"node-exporter-dnst6\" (UID: \"478b59a6-7c53-46ed-9a1d-944981dd4b7c\") " pod="openshift-monitoring/node-exporter-dnst6" Sep 10 02:50:35.706491 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.706138 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/478b59a6-7c53-46ed-9a1d-944981dd4b7c-node-exporter-tls\") pod \"node-exporter-dnst6\" (UID: \"478b59a6-7c53-46ed-9a1d-944981dd4b7c\") " pod="openshift-monitoring/node-exporter-dnst6" Sep 10 02:50:35.706491 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.706201 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/478b59a6-7c53-46ed-9a1d-944981dd4b7c-node-exporter-wtmp\") pod \"node-exporter-dnst6\" (UID: \"478b59a6-7c53-46ed-9a1d-944981dd4b7c\") " pod="openshift-monitoring/node-exporter-dnst6" Sep 10 02:50:35.706491 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.706222 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/478b59a6-7c53-46ed-9a1d-944981dd4b7c-metrics-client-ca\") pod \"node-exporter-dnst6\" (UID: \"478b59a6-7c53-46ed-9a1d-944981dd4b7c\") " pod="openshift-monitoring/node-exporter-dnst6" Sep 10 02:50:35.706491 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.706360 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5x5f8\" (UniqueName: \"kubernetes.io/projected/478b59a6-7c53-46ed-9a1d-944981dd4b7c-kube-api-access-5x5f8\") pod \"node-exporter-dnst6\" (UID: \"478b59a6-7c53-46ed-9a1d-944981dd4b7c\") " pod="openshift-monitoring/node-exporter-dnst6" Sep 10 02:50:35.706491 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.706464 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/478b59a6-7c53-46ed-9a1d-944981dd4b7c-root\") pod \"node-exporter-dnst6\" (UID: \"478b59a6-7c53-46ed-9a1d-944981dd4b7c\") " pod="openshift-monitoring/node-exporter-dnst6" Sep 10 02:50:35.706844 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.706541 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/478b59a6-7c53-46ed-9a1d-944981dd4b7c-root\") pod \"node-exporter-dnst6\" (UID: \"478b59a6-7c53-46ed-9a1d-944981dd4b7c\") " pod="openshift-monitoring/node-exporter-dnst6" Sep 10 02:50:35.706844 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.706467 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/478b59a6-7c53-46ed-9a1d-944981dd4b7c-node-exporter-textfile\") pod \"node-exporter-dnst6\" (UID: \"478b59a6-7c53-46ed-9a1d-944981dd4b7c\") " pod="openshift-monitoring/node-exporter-dnst6" Sep 10 02:50:35.706844 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.706363 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/478b59a6-7c53-46ed-9a1d-944981dd4b7c-node-exporter-wtmp\") pod \"node-exporter-dnst6\" (UID: \"478b59a6-7c53-46ed-9a1d-944981dd4b7c\") " pod="openshift-monitoring/node-exporter-dnst6" Sep 10 02:50:35.706844 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.706829 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/478b59a6-7c53-46ed-9a1d-944981dd4b7c-node-exporter-accelerators-collector-config\") pod \"node-exporter-dnst6\" (UID: \"478b59a6-7c53-46ed-9a1d-944981dd4b7c\") " pod="openshift-monitoring/node-exporter-dnst6" Sep 10 02:50:35.706981 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.706964 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/478b59a6-7c53-46ed-9a1d-944981dd4b7c-metrics-client-ca\") pod \"node-exporter-dnst6\" (UID: \"478b59a6-7c53-46ed-9a1d-944981dd4b7c\") " pod="openshift-monitoring/node-exporter-dnst6" Sep 10 02:50:35.708666 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.708643 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/478b59a6-7c53-46ed-9a1d-944981dd4b7c-node-exporter-tls\") pod \"node-exporter-dnst6\" (UID: \"478b59a6-7c53-46ed-9a1d-944981dd4b7c\") " pod="openshift-monitoring/node-exporter-dnst6" Sep 10 02:50:35.708752 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.708717 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/478b59a6-7c53-46ed-9a1d-944981dd4b7c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dnst6\" (UID: \"478b59a6-7c53-46ed-9a1d-944981dd4b7c\") " pod="openshift-monitoring/node-exporter-dnst6" Sep 10 02:50:35.715861 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.715841 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5x5f8\" (UniqueName: \"kubernetes.io/projected/478b59a6-7c53-46ed-9a1d-944981dd4b7c-kube-api-access-5x5f8\") pod \"node-exporter-dnst6\" (UID: \"478b59a6-7c53-46ed-9a1d-944981dd4b7c\") " pod="openshift-monitoring/node-exporter-dnst6" Sep 10 02:50:35.808925 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:35.808843 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dnst6" Sep 10 02:50:35.818769 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:50:35.818746 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod478b59a6_7c53_46ed_9a1d_944981dd4b7c.slice/crio-f34abd0e9be48ee3071dc6fad81f14108e2d050ad56858afd03bb57fcf8d9cdc WatchSource:0}: Error finding container f34abd0e9be48ee3071dc6fad81f14108e2d050ad56858afd03bb57fcf8d9cdc: Status 404 returned error can't find the container with id f34abd0e9be48ee3071dc6fad81f14108e2d050ad56858afd03bb57fcf8d9cdc Sep 10 02:50:36.086320 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:36.086219 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dnst6" event={"ID":"478b59a6-7c53-46ed-9a1d-944981dd4b7c","Type":"ContainerStarted","Data":"f34abd0e9be48ee3071dc6fad81f14108e2d050ad56858afd03bb57fcf8d9cdc"} Sep 10 02:50:36.088456 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:36.088416 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-nbnc9" event={"ID":"c5d95d1c-394b-4750-99c5-ff8ea4afefd1","Type":"ContainerStarted","Data":"e0f7ebb6f9ff552c2241c43826c034ccdffe8e59f78728fcddf14fe2c2f2a8ca"} Sep 10 02:50:37.095369 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:37.095329 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dnst6" event={"ID":"478b59a6-7c53-46ed-9a1d-944981dd4b7c","Type":"ContainerStarted","Data":"53514fedb9812510b44d95510323a24594257d14587347311f1db8c156e0b717"} Sep 10 02:50:38.101812 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:38.101774 2560 generic.go:356] "Generic (PLEG): container finished" podID="478b59a6-7c53-46ed-9a1d-944981dd4b7c" containerID="53514fedb9812510b44d95510323a24594257d14587347311f1db8c156e0b717" exitCode=0 Sep 10 02:50:38.102267 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:38.101834 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dnst6" event={"ID":"478b59a6-7c53-46ed-9a1d-944981dd4b7c","Type":"ContainerDied","Data":"53514fedb9812510b44d95510323a24594257d14587347311f1db8c156e0b717"} Sep 10 02:50:38.405267 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:38.405231 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-84ddd6465c-w8zvq"] Sep 10 02:50:38.408253 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:38.408231 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-84ddd6465c-w8zvq" Sep 10 02:50:38.414827 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:38.414486 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Sep 10 02:50:38.414827 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:38.414526 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-d5lg7\"" Sep 10 02:50:38.414827 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:38.414642 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-4ith6ipdasnpn\"" Sep 10 02:50:38.414827 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:38.414811 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Sep 10 02:50:38.415093 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:38.414812 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Sep 10 02:50:38.415093 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:38.414992 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Sep 10 02:50:38.415601 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:38.415581 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Sep 10 02:50:38.432524 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:38.432386 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4eb29f8f-bee4-445f-ac27-85be89b303a1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-84ddd6465c-w8zvq\" (UID: \"4eb29f8f-bee4-445f-ac27-85be89b303a1\") " pod="openshift-monitoring/thanos-querier-84ddd6465c-w8zvq" Sep 10 02:50:38.432524 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:38.432454 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4eb29f8f-bee4-445f-ac27-85be89b303a1-secret-thanos-querier-tls\") pod \"thanos-querier-84ddd6465c-w8zvq\" (UID: \"4eb29f8f-bee4-445f-ac27-85be89b303a1\") " pod="openshift-monitoring/thanos-querier-84ddd6465c-w8zvq" Sep 10 02:50:38.432524 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:38.432479 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4eb29f8f-bee4-445f-ac27-85be89b303a1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-84ddd6465c-w8zvq\" (UID: \"4eb29f8f-bee4-445f-ac27-85be89b303a1\") " pod="openshift-monitoring/thanos-querier-84ddd6465c-w8zvq" Sep 10 02:50:38.432772 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:38.432540 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4eb29f8f-bee4-445f-ac27-85be89b303a1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-84ddd6465c-w8zvq\" (UID: \"4eb29f8f-bee4-445f-ac27-85be89b303a1\") " pod="openshift-monitoring/thanos-querier-84ddd6465c-w8zvq" Sep 10 02:50:38.432772 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:38.432604 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4eb29f8f-bee4-445f-ac27-85be89b303a1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-84ddd6465c-w8zvq\" (UID: \"4eb29f8f-bee4-445f-ac27-85be89b303a1\") " pod="openshift-monitoring/thanos-querier-84ddd6465c-w8zvq" Sep 10 02:50:38.432772 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:38.432647 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xnn2\" (UniqueName: \"kubernetes.io/projected/4eb29f8f-bee4-445f-ac27-85be89b303a1-kube-api-access-4xnn2\") pod \"thanos-querier-84ddd6465c-w8zvq\" (UID: \"4eb29f8f-bee4-445f-ac27-85be89b303a1\") " pod="openshift-monitoring/thanos-querier-84ddd6465c-w8zvq" Sep 10 02:50:38.432772 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:38.432688 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4eb29f8f-bee4-445f-ac27-85be89b303a1-metrics-client-ca\") pod \"thanos-querier-84ddd6465c-w8zvq\" (UID: \"4eb29f8f-bee4-445f-ac27-85be89b303a1\") " pod="openshift-monitoring/thanos-querier-84ddd6465c-w8zvq" Sep 10 02:50:38.432772 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:38.432711 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4eb29f8f-bee4-445f-ac27-85be89b303a1-secret-grpc-tls\") pod \"thanos-querier-84ddd6465c-w8zvq\" (UID: \"4eb29f8f-bee4-445f-ac27-85be89b303a1\") " pod="openshift-monitoring/thanos-querier-84ddd6465c-w8zvq" Sep 10 02:50:38.434417 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:38.434399 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-84ddd6465c-w8zvq"] Sep 10 02:50:38.534150 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:38.534115 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4eb29f8f-bee4-445f-ac27-85be89b303a1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-84ddd6465c-w8zvq\" (UID: \"4eb29f8f-bee4-445f-ac27-85be89b303a1\") " pod="openshift-monitoring/thanos-querier-84ddd6465c-w8zvq" Sep 10 02:50:38.534337 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:38.534168 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xnn2\" (UniqueName: \"kubernetes.io/projected/4eb29f8f-bee4-445f-ac27-85be89b303a1-kube-api-access-4xnn2\") pod \"thanos-querier-84ddd6465c-w8zvq\" (UID: \"4eb29f8f-bee4-445f-ac27-85be89b303a1\") " pod="openshift-monitoring/thanos-querier-84ddd6465c-w8zvq" Sep 10 02:50:38.534337 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:38.534205 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4eb29f8f-bee4-445f-ac27-85be89b303a1-metrics-client-ca\") pod \"thanos-querier-84ddd6465c-w8zvq\" (UID: \"4eb29f8f-bee4-445f-ac27-85be89b303a1\") " pod="openshift-monitoring/thanos-querier-84ddd6465c-w8zvq" Sep 10 02:50:38.534337 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:38.534232 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4eb29f8f-bee4-445f-ac27-85be89b303a1-secret-grpc-tls\") pod \"thanos-querier-84ddd6465c-w8zvq\" (UID: \"4eb29f8f-bee4-445f-ac27-85be89b303a1\") " pod="openshift-monitoring/thanos-querier-84ddd6465c-w8zvq" Sep 10 02:50:38.534337 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:38.534273 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4eb29f8f-bee4-445f-ac27-85be89b303a1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-84ddd6465c-w8zvq\" (UID: \"4eb29f8f-bee4-445f-ac27-85be89b303a1\") " pod="openshift-monitoring/thanos-querier-84ddd6465c-w8zvq" Sep 10 02:50:38.534337 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:38.534327 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4eb29f8f-bee4-445f-ac27-85be89b303a1-secret-thanos-querier-tls\") pod \"thanos-querier-84ddd6465c-w8zvq\" (UID: \"4eb29f8f-bee4-445f-ac27-85be89b303a1\") " pod="openshift-monitoring/thanos-querier-84ddd6465c-w8zvq" Sep 10 02:50:38.534547 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:38.534357 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4eb29f8f-bee4-445f-ac27-85be89b303a1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-84ddd6465c-w8zvq\" (UID: \"4eb29f8f-bee4-445f-ac27-85be89b303a1\") " pod="openshift-monitoring/thanos-querier-84ddd6465c-w8zvq" Sep 10 02:50:38.534547 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:38.534399 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4eb29f8f-bee4-445f-ac27-85be89b303a1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-84ddd6465c-w8zvq\" (UID: \"4eb29f8f-bee4-445f-ac27-85be89b303a1\") " pod="openshift-monitoring/thanos-querier-84ddd6465c-w8zvq" Sep 10 02:50:38.535450 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:38.535319 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4eb29f8f-bee4-445f-ac27-85be89b303a1-metrics-client-ca\") pod \"thanos-querier-84ddd6465c-w8zvq\" (UID: \"4eb29f8f-bee4-445f-ac27-85be89b303a1\") " pod="openshift-monitoring/thanos-querier-84ddd6465c-w8zvq" Sep 10 02:50:38.537799 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:38.537777 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4eb29f8f-bee4-445f-ac27-85be89b303a1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-84ddd6465c-w8zvq\" (UID: \"4eb29f8f-bee4-445f-ac27-85be89b303a1\") " pod="openshift-monitoring/thanos-querier-84ddd6465c-w8zvq" Sep 10 02:50:38.538031 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:38.538004 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4eb29f8f-bee4-445f-ac27-85be89b303a1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-84ddd6465c-w8zvq\" (UID: \"4eb29f8f-bee4-445f-ac27-85be89b303a1\") " pod="openshift-monitoring/thanos-querier-84ddd6465c-w8zvq" Sep 10 02:50:38.538119 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:38.538017 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4eb29f8f-bee4-445f-ac27-85be89b303a1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-84ddd6465c-w8zvq\" (UID: \"4eb29f8f-bee4-445f-ac27-85be89b303a1\") " pod="openshift-monitoring/thanos-querier-84ddd6465c-w8zvq" Sep 10 02:50:38.538176 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:38.538132 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4eb29f8f-bee4-445f-ac27-85be89b303a1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-84ddd6465c-w8zvq\" (UID: \"4eb29f8f-bee4-445f-ac27-85be89b303a1\") " pod="openshift-monitoring/thanos-querier-84ddd6465c-w8zvq" Sep 10 02:50:38.538358 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:38.538331 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4eb29f8f-bee4-445f-ac27-85be89b303a1-secret-thanos-querier-tls\") pod \"thanos-querier-84ddd6465c-w8zvq\" (UID: \"4eb29f8f-bee4-445f-ac27-85be89b303a1\") " pod="openshift-monitoring/thanos-querier-84ddd6465c-w8zvq" Sep 10 02:50:38.538815 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:38.538797 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4eb29f8f-bee4-445f-ac27-85be89b303a1-secret-grpc-tls\") pod \"thanos-querier-84ddd6465c-w8zvq\" (UID: \"4eb29f8f-bee4-445f-ac27-85be89b303a1\") " pod="openshift-monitoring/thanos-querier-84ddd6465c-w8zvq" Sep 10 02:50:38.543968 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:38.543947 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xnn2\" (UniqueName: \"kubernetes.io/projected/4eb29f8f-bee4-445f-ac27-85be89b303a1-kube-api-access-4xnn2\") pod \"thanos-querier-84ddd6465c-w8zvq\" (UID: \"4eb29f8f-bee4-445f-ac27-85be89b303a1\") " pod="openshift-monitoring/thanos-querier-84ddd6465c-w8zvq" Sep 10 02:50:38.721366 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:38.721282 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-84ddd6465c-w8zvq" Sep 10 02:50:38.960952 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:38.960919 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-fqfck" Sep 10 02:50:39.011979 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:39.011899 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-84ddd6465c-w8zvq"] Sep 10 02:50:39.016848 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:50:39.016806 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4eb29f8f_bee4_445f_ac27_85be89b303a1.slice/crio-9fe402d376acf48953911998f6be70306d1dd9a5d63db354f82316f98c63660b WatchSource:0}: Error finding container 9fe402d376acf48953911998f6be70306d1dd9a5d63db354f82316f98c63660b: Status 404 returned error can't find the container with id 9fe402d376acf48953911998f6be70306d1dd9a5d63db354f82316f98c63660b Sep 10 02:50:39.108725 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:39.108614 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dnst6" event={"ID":"478b59a6-7c53-46ed-9a1d-944981dd4b7c","Type":"ContainerStarted","Data":"f29eb5d7941b1571d4ffe3996223210ae134085db234a76715370b3b726198c8"} Sep 10 02:50:39.108725 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:39.108683 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dnst6" event={"ID":"478b59a6-7c53-46ed-9a1d-944981dd4b7c","Type":"ContainerStarted","Data":"2d9a72480a805772915299bafbc0d66f8c2ea2499e4612dd2b9f648bc497e97c"} Sep 10 02:50:39.109966 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:39.109933 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84ddd6465c-w8zvq" event={"ID":"4eb29f8f-bee4-445f-ac27-85be89b303a1","Type":"ContainerStarted","Data":"9fe402d376acf48953911998f6be70306d1dd9a5d63db354f82316f98c63660b"} Sep 10 02:50:39.131670 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:39.131609 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-dnst6" podStartSLOduration=2.923782682 podStartE2EDuration="4.131584563s" totalImagesPullingTime="1.128586424s" totalInitContainerRuntime="79.215457ms" isStatefulPod=true podCreationTimestamp="2026-09-10 02:50:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:50:39.130195833 +0000 UTC m=+97.977025101" watchObservedRunningTime="2026-09-10 02:50:39.131584563 +0000 UTC m=+97.978413832" Sep 10 02:50:39.916343 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:39.916304 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-sq2zn"] Sep 10 02:50:39.919173 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:39.919150 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-sq2zn" Sep 10 02:50:39.922346 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:39.922313 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-8npjf\"" Sep 10 02:50:39.922574 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:39.922553 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Sep 10 02:50:39.936027 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:39.936000 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-sq2zn"] Sep 10 02:50:39.946789 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:39.946762 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hrrg\" (UniqueName: \"kubernetes.io/projected/99fb4a2a-f696-4312-8cc5-8123dfe4fc42-kube-api-access-8hrrg\") pod \"monitoring-plugin-85bbbbbf4-sq2zn\" (UID: \"99fb4a2a-f696-4312-8cc5-8123dfe4fc42\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-sq2zn" Sep 10 02:50:39.946917 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:39.946808 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/99fb4a2a-f696-4312-8cc5-8123dfe4fc42-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-sq2zn\" (UID: \"99fb4a2a-f696-4312-8cc5-8123dfe4fc42\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-sq2zn" Sep 10 02:50:40.047896 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.047854 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hrrg\" (UniqueName: \"kubernetes.io/projected/99fb4a2a-f696-4312-8cc5-8123dfe4fc42-kube-api-access-8hrrg\") pod \"monitoring-plugin-85bbbbbf4-sq2zn\" (UID: \"99fb4a2a-f696-4312-8cc5-8123dfe4fc42\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-sq2zn" Sep 10 02:50:40.048095 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.047913 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/99fb4a2a-f696-4312-8cc5-8123dfe4fc42-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-sq2zn\" (UID: \"99fb4a2a-f696-4312-8cc5-8123dfe4fc42\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-sq2zn" Sep 10 02:50:40.048095 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.047997 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/98643336-46ce-4f17-acb2-d8110a5bfb84-metrics-tls\") pod \"dns-default-2s6nr\" (UID: \"98643336-46ce-4f17-acb2-d8110a5bfb84\") " pod="openshift-dns/dns-default-2s6nr" Sep 10 02:50:40.048095 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:50:40.048034 2560 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Sep 10 02:50:40.048273 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:50:40.048103 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99fb4a2a-f696-4312-8cc5-8123dfe4fc42-monitoring-plugin-cert podName:99fb4a2a-f696-4312-8cc5-8123dfe4fc42 nodeName:}" failed. No retries permitted until 2026-09-10 02:50:40.548083652 +0000 UTC m=+99.394912912 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/99fb4a2a-f696-4312-8cc5-8123dfe4fc42-monitoring-plugin-cert") pod "monitoring-plugin-85bbbbbf4-sq2zn" (UID: "99fb4a2a-f696-4312-8cc5-8123dfe4fc42") : secret "monitoring-plugin-cert" not found Sep 10 02:50:40.048273 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.048040 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-registry-tls\") pod \"image-registry-75f8b47547-m9np9\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " pod="openshift-image-registry/image-registry-75f8b47547-m9np9" Sep 10 02:50:40.051204 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.051175 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/98643336-46ce-4f17-acb2-d8110a5bfb84-metrics-tls\") pod \"dns-default-2s6nr\" (UID: \"98643336-46ce-4f17-acb2-d8110a5bfb84\") " pod="openshift-dns/dns-default-2s6nr" Sep 10 02:50:40.051409 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.051386 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-registry-tls\") pod \"image-registry-75f8b47547-m9np9\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " pod="openshift-image-registry/image-registry-75f8b47547-m9np9" Sep 10 02:50:40.057840 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.057815 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hrrg\" (UniqueName: \"kubernetes.io/projected/99fb4a2a-f696-4312-8cc5-8123dfe4fc42-kube-api-access-8hrrg\") pod \"monitoring-plugin-85bbbbbf4-sq2zn\" (UID: \"99fb4a2a-f696-4312-8cc5-8123dfe4fc42\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-sq2zn" Sep 10 02:50:40.270055 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.269973 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-glllk\"" Sep 10 02:50:40.277683 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.277656 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75f8b47547-m9np9" Sep 10 02:50:40.291889 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.291861 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-nh6lc\"" Sep 10 02:50:40.299412 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.299387 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2s6nr" Sep 10 02:50:40.553298 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.553203 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/99fb4a2a-f696-4312-8cc5-8123dfe4fc42-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-sq2zn\" (UID: \"99fb4a2a-f696-4312-8cc5-8123dfe4fc42\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-sq2zn" Sep 10 02:50:40.556207 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.556180 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/99fb4a2a-f696-4312-8cc5-8123dfe4fc42-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-sq2zn\" (UID: \"99fb4a2a-f696-4312-8cc5-8123dfe4fc42\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-sq2zn" Sep 10 02:50:40.576715 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.576683 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-75f8b47547-m9np9"] Sep 10 02:50:40.596849 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.596759 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2s6nr"] Sep 10 02:50:40.728282 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.728247 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5dbb887c79-cbtkd"] Sep 10 02:50:40.731804 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.731776 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5dbb887c79-cbtkd" Sep 10 02:50:40.736479 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.736452 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-m87gv\"" Sep 10 02:50:40.736737 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.736716 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 10 02:50:40.736930 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.736911 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 10 02:50:40.737185 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.737171 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 10 02:50:40.737351 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.737338 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 10 02:50:40.738166 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.737994 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 10 02:50:40.743097 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.742907 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 10 02:50:40.755024 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.754994 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/34bee801-1833-4b3f-a078-7d1e798d9c94-telemeter-client-tls\") pod \"telemeter-client-5dbb887c79-cbtkd\" (UID: \"34bee801-1833-4b3f-a078-7d1e798d9c94\") " pod="openshift-monitoring/telemeter-client-5dbb887c79-cbtkd" Sep 10 02:50:40.755446 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.755046 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34bee801-1833-4b3f-a078-7d1e798d9c94-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5dbb887c79-cbtkd\" (UID: \"34bee801-1833-4b3f-a078-7d1e798d9c94\") " pod="openshift-monitoring/telemeter-client-5dbb887c79-cbtkd" Sep 10 02:50:40.755446 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.755113 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh9s7\" (UniqueName: \"kubernetes.io/projected/34bee801-1833-4b3f-a078-7d1e798d9c94-kube-api-access-xh9s7\") pod \"telemeter-client-5dbb887c79-cbtkd\" (UID: \"34bee801-1833-4b3f-a078-7d1e798d9c94\") " pod="openshift-monitoring/telemeter-client-5dbb887c79-cbtkd" Sep 10 02:50:40.755446 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.755194 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/34bee801-1833-4b3f-a078-7d1e798d9c94-secret-telemeter-client\") pod \"telemeter-client-5dbb887c79-cbtkd\" (UID: \"34bee801-1833-4b3f-a078-7d1e798d9c94\") " pod="openshift-monitoring/telemeter-client-5dbb887c79-cbtkd" Sep 10 02:50:40.755446 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.755262 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34bee801-1833-4b3f-a078-7d1e798d9c94-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5dbb887c79-cbtkd\" (UID: \"34bee801-1833-4b3f-a078-7d1e798d9c94\") " pod="openshift-monitoring/telemeter-client-5dbb887c79-cbtkd" Sep 10 02:50:40.755446 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.755295 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34bee801-1833-4b3f-a078-7d1e798d9c94-metrics-client-ca\") pod \"telemeter-client-5dbb887c79-cbtkd\" (UID: \"34bee801-1833-4b3f-a078-7d1e798d9c94\") " pod="openshift-monitoring/telemeter-client-5dbb887c79-cbtkd" Sep 10 02:50:40.755446 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.755362 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/34bee801-1833-4b3f-a078-7d1e798d9c94-federate-client-tls\") pod \"telemeter-client-5dbb887c79-cbtkd\" (UID: \"34bee801-1833-4b3f-a078-7d1e798d9c94\") " pod="openshift-monitoring/telemeter-client-5dbb887c79-cbtkd" Sep 10 02:50:40.755446 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.755412 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34bee801-1833-4b3f-a078-7d1e798d9c94-serving-certs-ca-bundle\") pod \"telemeter-client-5dbb887c79-cbtkd\" (UID: \"34bee801-1833-4b3f-a078-7d1e798d9c94\") " pod="openshift-monitoring/telemeter-client-5dbb887c79-cbtkd" Sep 10 02:50:40.757734 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.757708 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5dbb887c79-cbtkd"] Sep 10 02:50:40.834682 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.834606 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-sq2zn" Sep 10 02:50:40.856636 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.856596 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34bee801-1833-4b3f-a078-7d1e798d9c94-serving-certs-ca-bundle\") pod \"telemeter-client-5dbb887c79-cbtkd\" (UID: \"34bee801-1833-4b3f-a078-7d1e798d9c94\") " pod="openshift-monitoring/telemeter-client-5dbb887c79-cbtkd" Sep 10 02:50:40.856807 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.856670 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/34bee801-1833-4b3f-a078-7d1e798d9c94-telemeter-client-tls\") pod \"telemeter-client-5dbb887c79-cbtkd\" (UID: \"34bee801-1833-4b3f-a078-7d1e798d9c94\") " pod="openshift-monitoring/telemeter-client-5dbb887c79-cbtkd" Sep 10 02:50:40.856807 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.856721 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34bee801-1833-4b3f-a078-7d1e798d9c94-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5dbb887c79-cbtkd\" (UID: \"34bee801-1833-4b3f-a078-7d1e798d9c94\") " pod="openshift-monitoring/telemeter-client-5dbb887c79-cbtkd" Sep 10 02:50:40.856807 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.856751 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh9s7\" (UniqueName: \"kubernetes.io/projected/34bee801-1833-4b3f-a078-7d1e798d9c94-kube-api-access-xh9s7\") pod \"telemeter-client-5dbb887c79-cbtkd\" (UID: \"34bee801-1833-4b3f-a078-7d1e798d9c94\") " pod="openshift-monitoring/telemeter-client-5dbb887c79-cbtkd" Sep 10 02:50:40.856807 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.856785 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/34bee801-1833-4b3f-a078-7d1e798d9c94-secret-telemeter-client\") pod \"telemeter-client-5dbb887c79-cbtkd\" (UID: \"34bee801-1833-4b3f-a078-7d1e798d9c94\") " pod="openshift-monitoring/telemeter-client-5dbb887c79-cbtkd" Sep 10 02:50:40.857031 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.856807 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34bee801-1833-4b3f-a078-7d1e798d9c94-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5dbb887c79-cbtkd\" (UID: \"34bee801-1833-4b3f-a078-7d1e798d9c94\") " pod="openshift-monitoring/telemeter-client-5dbb887c79-cbtkd" Sep 10 02:50:40.857031 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.856834 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34bee801-1833-4b3f-a078-7d1e798d9c94-metrics-client-ca\") pod \"telemeter-client-5dbb887c79-cbtkd\" (UID: \"34bee801-1833-4b3f-a078-7d1e798d9c94\") " pod="openshift-monitoring/telemeter-client-5dbb887c79-cbtkd" Sep 10 02:50:40.857031 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.856890 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/34bee801-1833-4b3f-a078-7d1e798d9c94-federate-client-tls\") pod \"telemeter-client-5dbb887c79-cbtkd\" (UID: \"34bee801-1833-4b3f-a078-7d1e798d9c94\") " pod="openshift-monitoring/telemeter-client-5dbb887c79-cbtkd" Sep 10 02:50:40.858372 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.857427 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34bee801-1833-4b3f-a078-7d1e798d9c94-serving-certs-ca-bundle\") pod \"telemeter-client-5dbb887c79-cbtkd\" (UID: \"34bee801-1833-4b3f-a078-7d1e798d9c94\") " pod="openshift-monitoring/telemeter-client-5dbb887c79-cbtkd" Sep 10 02:50:40.858372 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.858327 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34bee801-1833-4b3f-a078-7d1e798d9c94-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5dbb887c79-cbtkd\" (UID: \"34bee801-1833-4b3f-a078-7d1e798d9c94\") " pod="openshift-monitoring/telemeter-client-5dbb887c79-cbtkd" Sep 10 02:50:40.858616 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.858578 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34bee801-1833-4b3f-a078-7d1e798d9c94-metrics-client-ca\") pod \"telemeter-client-5dbb887c79-cbtkd\" (UID: \"34bee801-1833-4b3f-a078-7d1e798d9c94\") " pod="openshift-monitoring/telemeter-client-5dbb887c79-cbtkd" Sep 10 02:50:40.860372 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.860326 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/34bee801-1833-4b3f-a078-7d1e798d9c94-federate-client-tls\") pod \"telemeter-client-5dbb887c79-cbtkd\" (UID: \"34bee801-1833-4b3f-a078-7d1e798d9c94\") " pod="openshift-monitoring/telemeter-client-5dbb887c79-cbtkd" Sep 10 02:50:40.861151 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.861129 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/34bee801-1833-4b3f-a078-7d1e798d9c94-telemeter-client-tls\") pod \"telemeter-client-5dbb887c79-cbtkd\" (UID: \"34bee801-1833-4b3f-a078-7d1e798d9c94\") " pod="openshift-monitoring/telemeter-client-5dbb887c79-cbtkd" Sep 10 02:50:40.861242 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.861198 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/34bee801-1833-4b3f-a078-7d1e798d9c94-secret-telemeter-client\") pod \"telemeter-client-5dbb887c79-cbtkd\" (UID: \"34bee801-1833-4b3f-a078-7d1e798d9c94\") " pod="openshift-monitoring/telemeter-client-5dbb887c79-cbtkd" Sep 10 02:50:40.861805 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.861785 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34bee801-1833-4b3f-a078-7d1e798d9c94-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5dbb887c79-cbtkd\" (UID: \"34bee801-1833-4b3f-a078-7d1e798d9c94\") " pod="openshift-monitoring/telemeter-client-5dbb887c79-cbtkd" Sep 10 02:50:40.867011 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:40.866987 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh9s7\" (UniqueName: \"kubernetes.io/projected/34bee801-1833-4b3f-a078-7d1e798d9c94-kube-api-access-xh9s7\") pod \"telemeter-client-5dbb887c79-cbtkd\" (UID: \"34bee801-1833-4b3f-a078-7d1e798d9c94\") " pod="openshift-monitoring/telemeter-client-5dbb887c79-cbtkd" Sep 10 02:50:41.044300 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.044255 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5dbb887c79-cbtkd" Sep 10 02:50:41.057945 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:50:41.057907 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98643336_46ce_4f17_acb2_d8110a5bfb84.slice/crio-d66ea6dae3e61296359450552e5f4aa5cbabe371785aff2bcd5097fc35a64833 WatchSource:0}: Error finding container d66ea6dae3e61296359450552e5f4aa5cbabe371785aff2bcd5097fc35a64833: Status 404 returned error can't find the container with id d66ea6dae3e61296359450552e5f4aa5cbabe371785aff2bcd5097fc35a64833 Sep 10 02:50:41.059200 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:50:41.059178 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4d56e77_0fdc_4733_b06f_2451d59f8343.slice/crio-443ec75639e8c28072c7d6db7a75d0c038bd3a6b42676198df5d589ecc1c2728 WatchSource:0}: Error finding container 443ec75639e8c28072c7d6db7a75d0c038bd3a6b42676198df5d589ecc1c2728: Status 404 returned error can't find the container with id 443ec75639e8c28072c7d6db7a75d0c038bd3a6b42676198df5d589ecc1c2728 Sep 10 02:50:41.122770 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.121815 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2s6nr" event={"ID":"98643336-46ce-4f17-acb2-d8110a5bfb84","Type":"ContainerStarted","Data":"d66ea6dae3e61296359450552e5f4aa5cbabe371785aff2bcd5097fc35a64833"} Sep 10 02:50:41.128681 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.125363 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75f8b47547-m9np9" event={"ID":"e4d56e77-0fdc-4733-b06f-2451d59f8343","Type":"ContainerStarted","Data":"443ec75639e8c28072c7d6db7a75d0c038bd3a6b42676198df5d589ecc1c2728"} Sep 10 02:50:41.354783 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.354761 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-sq2zn"] Sep 10 02:50:41.357560 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:50:41.357044 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99fb4a2a_f696_4312_8cc5_8123dfe4fc42.slice/crio-5ea0e89be4b19117631fbfa38d1ac11b124cda733530613701aef3f002cea8ad WatchSource:0}: Error finding container 5ea0e89be4b19117631fbfa38d1ac11b124cda733530613701aef3f002cea8ad: Status 404 returned error can't find the container with id 5ea0e89be4b19117631fbfa38d1ac11b124cda733530613701aef3f002cea8ad Sep 10 02:50:41.365069 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.365039 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5dbb887c79-cbtkd"] Sep 10 02:50:41.369974 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:50:41.369946 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34bee801_1833_4b3f_a078_7d1e798d9c94.slice/crio-31c44e7b44293f306f04b5a3c49123aad6f20aff1ac1354cb58f5b2ad873d8c6 WatchSource:0}: Error finding container 31c44e7b44293f306f04b5a3c49123aad6f20aff1ac1354cb58f5b2ad873d8c6: Status 404 returned error can't find the container with id 31c44e7b44293f306f04b5a3c49123aad6f20aff1ac1354cb58f5b2ad873d8c6 Sep 10 02:50:41.608039 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.608007 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 02:50:41.612889 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.612861 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.620441 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.620130 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 10 02:50:41.620441 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.620183 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 10 02:50:41.620441 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.620231 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 10 02:50:41.620441 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.620187 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 10 02:50:41.623784 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.621123 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-rx4lc\"" Sep 10 02:50:41.623784 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.621312 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 10 02:50:41.623784 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.621498 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-h92momfi5tiu\"" Sep 10 02:50:41.623784 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.621711 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 10 02:50:41.623784 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.621923 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 10 02:50:41.623784 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.622131 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 10 02:50:41.623784 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.622172 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 10 02:50:41.623784 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.622132 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 10 02:50:41.623784 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.622358 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 10 02:50:41.623784 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.622583 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 10 02:50:41.623784 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.623082 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 10 02:50:41.625775 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.625753 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 10 02:50:41.627566 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.627540 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 10 02:50:41.647358 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.647330 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 02:50:41.666305 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.666266 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-config\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.666468 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.666314 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.666468 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.666341 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.666468 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.666368 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mjd7\" (UniqueName: \"kubernetes.io/projected/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-kube-api-access-5mjd7\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.666468 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.666396 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-web-config\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.666711 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.666484 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.666711 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.666542 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.666711 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.666569 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.666711 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.666607 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.666711 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.666637 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.666711 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.666693 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-config-out\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.667010 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.666753 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.667010 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.666781 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.667010 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.666804 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.667010 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.666866 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.667010 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.666898 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.667010 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.666920 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.667010 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.666953 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.667010 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.667003 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.667382 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.667029 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.770110 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.769078 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.770110 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.769119 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.770110 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.769141 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.770110 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.769173 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.770110 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.769212 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.770110 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.769233 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.770110 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.769274 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-config\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.770110 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.769298 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.770110 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.769325 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.770110 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.769350 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mjd7\" (UniqueName: \"kubernetes.io/projected/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-kube-api-access-5mjd7\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.770110 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.769380 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-web-config\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.770110 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.769427 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.770110 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.769457 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.770110 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.769480 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.770110 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.769534 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.770110 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.769565 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.770110 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.769604 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-config-out\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.771010 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.769642 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.775547 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.772727 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.775547 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.772765 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.775547 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.773179 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.775547 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.773446 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.775547 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.773539 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.775547 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.773928 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.775947 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.775568 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.776743 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.776304 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.776743 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.776359 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.776743 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.776694 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.777693 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.777669 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.778891 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.778850 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.779530 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.779484 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.779614 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.779581 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-config\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.779738 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.779679 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.779909 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.779891 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-web-config\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.781197 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.780534 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.787298 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.787273 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.787796 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.787689 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.788160 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.788134 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.788405 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.788385 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-config-out\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.792298 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.792203 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mjd7\" (UniqueName: \"kubernetes.io/projected/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-kube-api-access-5mjd7\") pod \"prometheus-k8s-0\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:41.933912 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:41.933576 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:50:42.135190 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:42.135150 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84ddd6465c-w8zvq" event={"ID":"4eb29f8f-bee4-445f-ac27-85be89b303a1","Type":"ContainerStarted","Data":"475fe9893e6933d2f81969d63488ef23f97a3d7e8fbc6f0319fdb572f38ad22a"} Sep 10 02:50:42.135374 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:42.135279 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84ddd6465c-w8zvq" event={"ID":"4eb29f8f-bee4-445f-ac27-85be89b303a1","Type":"ContainerStarted","Data":"7670d0e8bb1e590e9d330119c0e6fd710ce00ab760595e4d8c2215ebdfff66f6"} Sep 10 02:50:42.135374 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:42.135320 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84ddd6465c-w8zvq" event={"ID":"4eb29f8f-bee4-445f-ac27-85be89b303a1","Type":"ContainerStarted","Data":"c4ad8367e6db8a8c9a9e2587291fe85fb8331044b85d5a89b7832fcfa97620b6"} Sep 10 02:50:42.137849 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:42.137478 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75f8b47547-m9np9" event={"ID":"e4d56e77-0fdc-4733-b06f-2451d59f8343","Type":"ContainerStarted","Data":"9cf13538624022a9aea16770a8e8e4f3f887803ab039022a21062ae84b5eaf86"} Sep 10 02:50:42.137849 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:42.137582 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-75f8b47547-m9np9" Sep 10 02:50:42.140444 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:42.140413 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5dbb887c79-cbtkd" event={"ID":"34bee801-1833-4b3f-a078-7d1e798d9c94","Type":"ContainerStarted","Data":"31c44e7b44293f306f04b5a3c49123aad6f20aff1ac1354cb58f5b2ad873d8c6"} Sep 10 02:50:42.142134 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:42.142109 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-sq2zn" event={"ID":"99fb4a2a-f696-4312-8cc5-8123dfe4fc42","Type":"ContainerStarted","Data":"5ea0e89be4b19117631fbfa38d1ac11b124cda733530613701aef3f002cea8ad"} Sep 10 02:50:42.170490 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:42.169846 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-75f8b47547-m9np9" podStartSLOduration=99.169823961 podStartE2EDuration="1m39.169823961s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 02:49:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:50:42.168823734 +0000 UTC m=+101.015653015" watchObservedRunningTime="2026-09-10 02:50:42.169823961 +0000 UTC m=+101.016653228" Sep 10 02:50:42.247359 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:42.247280 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 02:50:42.256314 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:50:42.256278 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9af6a78_5e1f_4f97_9426_9ca4ad6d4527.slice/crio-cd9c174415d550c32f8ad568299b3575ae8e266d5f1c9cd75da2814c85184b48 WatchSource:0}: Error finding container cd9c174415d550c32f8ad568299b3575ae8e266d5f1c9cd75da2814c85184b48: Status 404 returned error can't find the container with id cd9c174415d550c32f8ad568299b3575ae8e266d5f1c9cd75da2814c85184b48 Sep 10 02:50:43.151297 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:43.151208 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527","Type":"ContainerStarted","Data":"cd9c174415d550c32f8ad568299b3575ae8e266d5f1c9cd75da2814c85184b48"} Sep 10 02:50:44.069851 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:44.069815 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-75f8b47547-m9np9"] Sep 10 02:50:45.167811 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:45.167775 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84ddd6465c-w8zvq" event={"ID":"4eb29f8f-bee4-445f-ac27-85be89b303a1","Type":"ContainerStarted","Data":"fa9ce1c364a2b0681377f75d7d94561461911d236702e9395f72edf203e1c1c6"} Sep 10 02:50:45.168168 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:45.167881 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84ddd6465c-w8zvq" event={"ID":"4eb29f8f-bee4-445f-ac27-85be89b303a1","Type":"ContainerStarted","Data":"5327b25cfc68faf9c8703ebfdd45fc02f64da793b26b97e56205c53d180c3b3c"} Sep 10 02:50:45.171245 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:45.171037 2560 generic.go:356] "Generic (PLEG): container finished" podID="c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" containerID="e098823bc59cae2a114b522b7183fed3a9a50ab6a72f6665eeb10ba156d96d1a" exitCode=0 Sep 10 02:50:45.171245 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:45.171164 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527","Type":"ContainerDied","Data":"e098823bc59cae2a114b522b7183fed3a9a50ab6a72f6665eeb10ba156d96d1a"} Sep 10 02:50:45.174743 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:45.174635 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5dbb887c79-cbtkd" event={"ID":"34bee801-1833-4b3f-a078-7d1e798d9c94","Type":"ContainerStarted","Data":"83d3666ebfc43f1a8f63e85c61f33d130e1c124ddc66268001fa1e54b12afe58"} Sep 10 02:50:45.174743 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:45.174701 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5dbb887c79-cbtkd" event={"ID":"34bee801-1833-4b3f-a078-7d1e798d9c94","Type":"ContainerStarted","Data":"b7124d16e2eabd0d8fad65353828214f3093add071dabeab386fdf9e0991ebc8"} Sep 10 02:50:45.178061 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:45.177593 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-sq2zn" event={"ID":"99fb4a2a-f696-4312-8cc5-8123dfe4fc42","Type":"ContainerStarted","Data":"affafc9468fb44783f3c1936660c84bbf582e72059de713c0fd101e4fe5ebb0a"} Sep 10 02:50:45.178061 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:45.177658 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-sq2zn" Sep 10 02:50:45.181047 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:45.181013 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2s6nr" event={"ID":"98643336-46ce-4f17-acb2-d8110a5bfb84","Type":"ContainerStarted","Data":"34deedf33aa78628f58b054fb1fd711aa3dce062295257357abdf262a0df7ab3"} Sep 10 02:50:45.181158 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:45.181063 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2s6nr" event={"ID":"98643336-46ce-4f17-acb2-d8110a5bfb84","Type":"ContainerStarted","Data":"95ebd68608174e673b4199e13c336d359c90a4ffeffc982e1f2d13fec120e952"} Sep 10 02:50:45.181655 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:45.181636 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2s6nr" Sep 10 02:50:45.183982 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:45.183942 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-sq2zn" Sep 10 02:50:45.225141 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:45.225089 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-sq2zn" podStartSLOduration=2.7840945440000002 podStartE2EDuration="6.225069408s" totalImagesPullingTime="3.440974864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 02:50:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:50:45.224307618 +0000 UTC m=+104.071136969" watchObservedRunningTime="2026-09-10 02:50:45.225069408 +0000 UTC m=+104.071898677" Sep 10 02:50:45.247274 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:45.247124 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2s6nr" podStartSLOduration=65.509270739 podStartE2EDuration="1m9.24709998s" totalImagesPullingTime="3.737829241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 02:49:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:50:45.246420261 +0000 UTC m=+104.093249531" watchObservedRunningTime="2026-09-10 02:50:45.24709998 +0000 UTC m=+104.093929248" Sep 10 02:50:46.188706 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:46.188665 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5dbb887c79-cbtkd" event={"ID":"34bee801-1833-4b3f-a078-7d1e798d9c94","Type":"ContainerStarted","Data":"d6c0a8e92516a3bb91470e84bbbd0f58d23d64ff3eec3ef3a7a1df302f50adf9"} Sep 10 02:50:46.193465 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:46.193413 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84ddd6465c-w8zvq" event={"ID":"4eb29f8f-bee4-445f-ac27-85be89b303a1","Type":"ContainerStarted","Data":"d6705469ef529815eaceec22a5bf34438f24fa692cfc6b8218564d2f52575327"} Sep 10 02:50:46.215271 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:46.215217 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5dbb887c79-cbtkd" podStartSLOduration=2.78820688 podStartE2EDuration="6.215201609s" totalImagesPullingTime="3.426994729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 02:50:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:50:46.212808863 +0000 UTC m=+105.059638141" watchObservedRunningTime="2026-09-10 02:50:46.215201609 +0000 UTC m=+105.062030877" Sep 10 02:50:46.245522 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:46.244084 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-84ddd6465c-w8zvq" podStartSLOduration=2.718540807 podStartE2EDuration="8.244060489s" totalImagesPullingTime="5.525519682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 02:50:38 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:50:46.24286336 +0000 UTC m=+105.089692642" watchObservedRunningTime="2026-09-10 02:50:46.244060489 +0000 UTC m=+105.090889753" Sep 10 02:50:47.201230 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:47.201190 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-84ddd6465c-w8zvq" Sep 10 02:50:47.207053 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:47.207025 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-84ddd6465c-w8zvq" Sep 10 02:50:55.239338 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:55.238240 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-nbnc9" event={"ID":"c5d95d1c-394b-4750-99c5-ff8ea4afefd1","Type":"ContainerStarted","Data":"1fd6d924df4947c3c2f0cabec89544b2ec8d0fc2a6015d6918a858c02f93c69f"} Sep 10 02:50:55.239338 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:55.238967 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-nbnc9" Sep 10 02:50:55.242711 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:55.242662 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527","Type":"ContainerStarted","Data":"a9af9d98465b2ce5dafdae2f51bb0e79c85035520505457bf0bf4a20832d1654"} Sep 10 02:50:55.242822 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:55.242726 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527","Type":"ContainerStarted","Data":"1268318acf5021553b7bcb34cd0dff82b770e3cf222ae746d8656d156edf13ed"} Sep 10 02:50:55.242822 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:55.242753 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527","Type":"ContainerStarted","Data":"eeb15fce8763984da6116a900bbd07f7809e344dceb4161b83c69194673a78a7"} Sep 10 02:50:55.242822 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:55.242790 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527","Type":"ContainerStarted","Data":"c2c484e5e7c0f002c0fc991b9f0e09855de7b861aad77e920c0789bebb0a548c"} Sep 10 02:50:55.249552 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:55.249530 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-nbnc9" Sep 10 02:50:55.260650 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:55.260581 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-nbnc9" podStartSLOduration=1.919558573 podStartE2EDuration="21.260565678s" totalImagesPullingTime="19.341007105s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 02:50:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:50:55.25769925 +0000 UTC m=+114.104528518" watchObservedRunningTime="2026-09-10 02:50:55.260565678 +0000 UTC m=+114.107394946" Sep 10 02:50:56.202968 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:56.202929 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2s6nr" Sep 10 02:50:56.252983 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:56.252942 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527","Type":"ContainerStarted","Data":"fdda688ef50676fd70c570e0f5f7326ddac84e603e94f0eace0bbb56139d4872"} Sep 10 02:50:56.253471 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:56.253030 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527","Type":"ContainerStarted","Data":"d5c8cde0f1d079a30d41f80c3aa0b0d462b03125fd4ea374dd2e2340e7beb70e"} Sep 10 02:50:56.289533 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:56.289446 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.98722678 podStartE2EDuration="15.289413978s" totalImagesPullingTime="12.265448059s" totalInitContainerRuntime="36.739139ms" isStatefulPod=false podCreationTimestamp="2026-09-10 02:50:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:50:56.287911183 +0000 UTC m=+115.134740474" watchObservedRunningTime="2026-09-10 02:50:56.289413978 +0000 UTC m=+115.136243248" Sep 10 02:50:56.934528 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:50:56.934465 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:51:04.164917 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:04.164884 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-75f8b47547-m9np9" Sep 10 02:51:08.309612 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:08.309579 2560 generic.go:356] "Generic (PLEG): container finished" podID="d06965d1-f868-4a05-a4a8-1878d2d8ced1" containerID="5be094f307d05149ea8401917ad20f920a790a6613e45dc9c420ddc8a6f20d5a" exitCode=0 Sep 10 02:51:08.309983 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:08.309653 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-dvdwf" event={"ID":"d06965d1-f868-4a05-a4a8-1878d2d8ced1","Type":"ContainerDied","Data":"5be094f307d05149ea8401917ad20f920a790a6613e45dc9c420ddc8a6f20d5a"} Sep 10 02:51:08.310064 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:08.310048 2560 scope.go:117] "RemoveContainer" containerID="5be094f307d05149ea8401917ad20f920a790a6613e45dc9c420ddc8a6f20d5a" Sep 10 02:51:08.311406 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:08.311383 2560 generic.go:356] "Generic (PLEG): container finished" podID="0a4e6681-5d69-48a7-b8e7-782f3e699b0a" containerID="a99d3e705a6dc157e49ab7d16802f1599b3812a07fdda2c9c15d492791e0ef39" exitCode=0 Sep 10 02:51:08.311555 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:08.311430 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-mpwt5" event={"ID":"0a4e6681-5d69-48a7-b8e7-782f3e699b0a","Type":"ContainerDied","Data":"a99d3e705a6dc157e49ab7d16802f1599b3812a07fdda2c9c15d492791e0ef39"} Sep 10 02:51:08.311749 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:08.311732 2560 scope.go:117] "RemoveContainer" containerID="a99d3e705a6dc157e49ab7d16802f1599b3812a07fdda2c9c15d492791e0ef39" Sep 10 02:51:09.182665 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:09.182623 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-75f8b47547-m9np9" podUID="e4d56e77-0fdc-4733-b06f-2451d59f8343" containerName="registry" containerID="cri-o://9cf13538624022a9aea16770a8e8e4f3f887803ab039022a21062ae84b5eaf86" gracePeriod=30 Sep 10 02:51:09.324137 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:09.324108 2560 generic.go:356] "Generic (PLEG): container finished" podID="e4d56e77-0fdc-4733-b06f-2451d59f8343" containerID="9cf13538624022a9aea16770a8e8e4f3f887803ab039022a21062ae84b5eaf86" exitCode=0 Sep 10 02:51:09.324569 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:09.324187 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75f8b47547-m9np9" event={"ID":"e4d56e77-0fdc-4733-b06f-2451d59f8343","Type":"ContainerDied","Data":"9cf13538624022a9aea16770a8e8e4f3f887803ab039022a21062ae84b5eaf86"} Sep 10 02:51:09.326276 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:09.326252 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-mpwt5" event={"ID":"0a4e6681-5d69-48a7-b8e7-782f3e699b0a","Type":"ContainerStarted","Data":"1f7949fdc56ebecc13259ca56c9c5e4a974f7e5a0c2c3bdc4d8501d2e0a40f5e"} Sep 10 02:51:09.328255 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:09.328234 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-dvdwf" event={"ID":"d06965d1-f868-4a05-a4a8-1878d2d8ced1","Type":"ContainerStarted","Data":"7cf5605b0d4e521f84d5f0bc9b78a214265c3f09d59ee4948b4535228466aa7d"} Sep 10 02:51:09.470121 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:09.470093 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75f8b47547-m9np9" Sep 10 02:51:09.650521 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:09.650489 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-bound-sa-token\") pod \"e4d56e77-0fdc-4733-b06f-2451d59f8343\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " Sep 10 02:51:09.650684 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:09.650579 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-registry-tls\") pod \"e4d56e77-0fdc-4733-b06f-2451d59f8343\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " Sep 10 02:51:09.650684 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:09.650632 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gnc2k\" (UniqueName: \"kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-kube-api-access-gnc2k\") pod \"e4d56e77-0fdc-4733-b06f-2451d59f8343\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " Sep 10 02:51:09.650684 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:09.650666 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e4d56e77-0fdc-4733-b06f-2451d59f8343-installation-pull-secrets\") pod \"e4d56e77-0fdc-4733-b06f-2451d59f8343\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " Sep 10 02:51:09.650829 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:09.650694 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e4d56e77-0fdc-4733-b06f-2451d59f8343-image-registry-private-configuration\") pod \"e4d56e77-0fdc-4733-b06f-2451d59f8343\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " Sep 10 02:51:09.650829 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:09.650717 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e4d56e77-0fdc-4733-b06f-2451d59f8343-registry-certificates\") pod \"e4d56e77-0fdc-4733-b06f-2451d59f8343\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " Sep 10 02:51:09.650926 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:09.650860 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e4d56e77-0fdc-4733-b06f-2451d59f8343-trusted-ca\") pod \"e4d56e77-0fdc-4733-b06f-2451d59f8343\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " Sep 10 02:51:09.651193 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:09.651168 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4d56e77-0fdc-4733-b06f-2451d59f8343-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "e4d56e77-0fdc-4733-b06f-2451d59f8343" (UID: "e4d56e77-0fdc-4733-b06f-2451d59f8343"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 02:51:09.651481 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:09.651411 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4d56e77-0fdc-4733-b06f-2451d59f8343-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "e4d56e77-0fdc-4733-b06f-2451d59f8343" (UID: "e4d56e77-0fdc-4733-b06f-2451d59f8343"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 02:51:09.651481 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:09.651476 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e4d56e77-0fdc-4733-b06f-2451d59f8343-ca-trust-extracted\") pod \"e4d56e77-0fdc-4733-b06f-2451d59f8343\" (UID: \"e4d56e77-0fdc-4733-b06f-2451d59f8343\") " Sep 10 02:51:09.651960 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:09.651938 2560 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e4d56e77-0fdc-4733-b06f-2451d59f8343-registry-certificates\") on node \"ip-10-0-133-37.ec2.internal\" DevicePath \"\"" Sep 10 02:51:09.652038 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:09.651965 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e4d56e77-0fdc-4733-b06f-2451d59f8343-trusted-ca\") on node \"ip-10-0-133-37.ec2.internal\" DevicePath \"\"" Sep 10 02:51:09.653349 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:09.653322 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e4d56e77-0fdc-4733-b06f-2451d59f8343" (UID: "e4d56e77-0fdc-4733-b06f-2451d59f8343"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 02:51:09.653349 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:09.653333 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4d56e77-0fdc-4733-b06f-2451d59f8343-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "e4d56e77-0fdc-4733-b06f-2451d59f8343" (UID: "e4d56e77-0fdc-4733-b06f-2451d59f8343"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 02:51:09.653538 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:09.653367 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "e4d56e77-0fdc-4733-b06f-2451d59f8343" (UID: "e4d56e77-0fdc-4733-b06f-2451d59f8343"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 02:51:09.653591 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:09.653536 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-kube-api-access-gnc2k" (OuterVolumeSpecName: "kube-api-access-gnc2k") pod "e4d56e77-0fdc-4733-b06f-2451d59f8343" (UID: "e4d56e77-0fdc-4733-b06f-2451d59f8343"). InnerVolumeSpecName "kube-api-access-gnc2k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 02:51:09.653628 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:09.653595 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4d56e77-0fdc-4733-b06f-2451d59f8343-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "e4d56e77-0fdc-4733-b06f-2451d59f8343" (UID: "e4d56e77-0fdc-4733-b06f-2451d59f8343"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 02:51:09.659590 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:09.659566 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4d56e77-0fdc-4733-b06f-2451d59f8343-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "e4d56e77-0fdc-4733-b06f-2451d59f8343" (UID: "e4d56e77-0fdc-4733-b06f-2451d59f8343"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 02:51:09.753342 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:09.753245 2560 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e4d56e77-0fdc-4733-b06f-2451d59f8343-ca-trust-extracted\") on node \"ip-10-0-133-37.ec2.internal\" DevicePath \"\"" Sep 10 02:51:09.753342 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:09.753284 2560 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-bound-sa-token\") on node \"ip-10-0-133-37.ec2.internal\" DevicePath \"\"" Sep 10 02:51:09.753342 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:09.753299 2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-registry-tls\") on node \"ip-10-0-133-37.ec2.internal\" DevicePath \"\"" Sep 10 02:51:09.753342 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:09.753310 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gnc2k\" (UniqueName: \"kubernetes.io/projected/e4d56e77-0fdc-4733-b06f-2451d59f8343-kube-api-access-gnc2k\") on node \"ip-10-0-133-37.ec2.internal\" DevicePath \"\"" Sep 10 02:51:09.753342 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:09.753324 2560 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e4d56e77-0fdc-4733-b06f-2451d59f8343-installation-pull-secrets\") on node \"ip-10-0-133-37.ec2.internal\" DevicePath \"\"" Sep 10 02:51:09.753342 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:09.753338 2560 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e4d56e77-0fdc-4733-b06f-2451d59f8343-image-registry-private-configuration\") on node \"ip-10-0-133-37.ec2.internal\" DevicePath \"\"" Sep 10 02:51:10.334216 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:10.334185 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75f8b47547-m9np9" Sep 10 02:51:10.334861 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:10.334223 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75f8b47547-m9np9" event={"ID":"e4d56e77-0fdc-4733-b06f-2451d59f8343","Type":"ContainerDied","Data":"443ec75639e8c28072c7d6db7a75d0c038bd3a6b42676198df5d589ecc1c2728"} Sep 10 02:51:10.334861 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:10.334286 2560 scope.go:117] "RemoveContainer" containerID="9cf13538624022a9aea16770a8e8e4f3f887803ab039022a21062ae84b5eaf86" Sep 10 02:51:10.356812 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:10.356782 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-75f8b47547-m9np9"] Sep 10 02:51:10.361282 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:10.361260 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-75f8b47547-m9np9"] Sep 10 02:51:11.470490 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:11.470447 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e-metrics-certs\") pod \"network-metrics-daemon-d2sbz\" (UID: \"c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e\") " pod="openshift-multus/network-metrics-daemon-d2sbz" Sep 10 02:51:11.472981 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:11.472947 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e-metrics-certs\") pod \"network-metrics-daemon-d2sbz\" (UID: \"c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e\") " pod="openshift-multus/network-metrics-daemon-d2sbz" Sep 10 02:51:11.645108 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:11.645073 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4d56e77-0fdc-4733-b06f-2451d59f8343" path="/var/lib/kubelet/pods/e4d56e77-0fdc-4733-b06f-2451d59f8343/volumes" Sep 10 02:51:11.752846 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:11.752771 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j5skv\"" Sep 10 02:51:11.760193 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:11.760174 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2sbz" Sep 10 02:51:12.069382 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:12.069350 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-d2sbz"] Sep 10 02:51:12.071868 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:51:12.071841 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5f72b58_af4e_4a2a_8b7f_24270dc8ef1e.slice/crio-38c2dc314d9e45299b2726502a143b1ffdf83c772025457ee6599c2b4a924e84 WatchSource:0}: Error finding container 38c2dc314d9e45299b2726502a143b1ffdf83c772025457ee6599c2b4a924e84: Status 404 returned error can't find the container with id 38c2dc314d9e45299b2726502a143b1ffdf83c772025457ee6599c2b4a924e84 Sep 10 02:51:12.346467 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:12.346387 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-d2sbz" event={"ID":"c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e","Type":"ContainerStarted","Data":"38c2dc314d9e45299b2726502a143b1ffdf83c772025457ee6599c2b4a924e84"} Sep 10 02:51:14.356592 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:14.356559 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-d2sbz" event={"ID":"c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e","Type":"ContainerStarted","Data":"2817c81422973cca0cb1673bc46bb9c4a513b0f07cc14691371b2b070a282bab"} Sep 10 02:51:14.357039 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:14.356625 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-d2sbz" event={"ID":"c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e","Type":"ContainerStarted","Data":"1f585cf776121bb55888b60c8800542bc988d4f322188802489436e4205c6e88"} Sep 10 02:51:14.382305 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:14.382256 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-d2sbz" podStartSLOduration=130.758611654 podStartE2EDuration="2m12.382238784s" totalImagesPullingTime="1.62362713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 02:49:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:51:14.3816649 +0000 UTC m=+133.228494181" watchObservedRunningTime="2026-09-10 02:51:14.382238784 +0000 UTC m=+133.229068052" Sep 10 02:51:28.416452 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:28.416416 2560 generic.go:356] "Generic (PLEG): container finished" podID="2badc582-316e-4b1e-b181-760ea2abbc08" containerID="43bf28409b57d912c132e06aac81b5665de04b8dd157e290be36282b8083c0e7" exitCode=0 Sep 10 02:51:28.416889 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:28.416473 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-jxcwr" event={"ID":"2badc582-316e-4b1e-b181-760ea2abbc08","Type":"ContainerDied","Data":"43bf28409b57d912c132e06aac81b5665de04b8dd157e290be36282b8083c0e7"} Sep 10 02:51:28.416889 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:28.416855 2560 scope.go:117] "RemoveContainer" containerID="43bf28409b57d912c132e06aac81b5665de04b8dd157e290be36282b8083c0e7" Sep 10 02:51:29.425086 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:29.425046 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-jxcwr" event={"ID":"2badc582-316e-4b1e-b181-760ea2abbc08","Type":"ContainerStarted","Data":"25de225d8a45b85a9c60a3ad9a455781cc7e1b85e498b132616fd7f85ca120b0"} Sep 10 02:51:41.934156 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:41.934117 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:51:41.952043 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:41.952001 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:51:42.500131 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:51:42.500099 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:00.059307 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:00.059213 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 02:52:00.059800 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:00.059684 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" containerName="prometheus" containerID="cri-o://c2c484e5e7c0f002c0fc991b9f0e09855de7b861aad77e920c0789bebb0a548c" gracePeriod=600 Sep 10 02:52:00.059800 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:00.059693 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" containerName="kube-rbac-proxy" containerID="cri-o://d5c8cde0f1d079a30d41f80c3aa0b0d462b03125fd4ea374dd2e2340e7beb70e" gracePeriod=600 Sep 10 02:52:00.059800 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:00.059724 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" containerName="kube-rbac-proxy-web" containerID="cri-o://a9af9d98465b2ce5dafdae2f51bb0e79c85035520505457bf0bf4a20832d1654" gracePeriod=600 Sep 10 02:52:00.059800 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:00.059694 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" containerName="kube-rbac-proxy-thanos" containerID="cri-o://fdda688ef50676fd70c570e0f5f7326ddac84e603e94f0eace0bbb56139d4872" gracePeriod=600 Sep 10 02:52:00.059998 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:00.059701 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" containerName="config-reloader" containerID="cri-o://eeb15fce8763984da6116a900bbd07f7809e344dceb4161b83c69194673a78a7" gracePeriod=600 Sep 10 02:52:00.059998 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:00.059694 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" containerName="thanos-sidecar" containerID="cri-o://1268318acf5021553b7bcb34cd0dff82b770e3cf222ae746d8656d156edf13ed" gracePeriod=600 Sep 10 02:52:00.566030 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:00.565994 2560 generic.go:356] "Generic (PLEG): container finished" podID="c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" containerID="fdda688ef50676fd70c570e0f5f7326ddac84e603e94f0eace0bbb56139d4872" exitCode=0 Sep 10 02:52:00.566030 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:00.566018 2560 generic.go:356] "Generic (PLEG): container finished" podID="c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" containerID="d5c8cde0f1d079a30d41f80c3aa0b0d462b03125fd4ea374dd2e2340e7beb70e" exitCode=0 Sep 10 02:52:00.566030 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:00.566024 2560 generic.go:356] "Generic (PLEG): container finished" podID="c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" containerID="1268318acf5021553b7bcb34cd0dff82b770e3cf222ae746d8656d156edf13ed" exitCode=0 Sep 10 02:52:00.566030 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:00.566029 2560 generic.go:356] "Generic (PLEG): container finished" podID="c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" containerID="eeb15fce8763984da6116a900bbd07f7809e344dceb4161b83c69194673a78a7" exitCode=0 Sep 10 02:52:00.566030 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:00.566034 2560 generic.go:356] "Generic (PLEG): container finished" podID="c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" containerID="c2c484e5e7c0f002c0fc991b9f0e09855de7b861aad77e920c0789bebb0a548c" exitCode=0 Sep 10 02:52:00.566395 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:00.566071 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527","Type":"ContainerDied","Data":"fdda688ef50676fd70c570e0f5f7326ddac84e603e94f0eace0bbb56139d4872"} Sep 10 02:52:00.566395 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:00.566153 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527","Type":"ContainerDied","Data":"d5c8cde0f1d079a30d41f80c3aa0b0d462b03125fd4ea374dd2e2340e7beb70e"} Sep 10 02:52:00.566395 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:00.566193 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527","Type":"ContainerDied","Data":"1268318acf5021553b7bcb34cd0dff82b770e3cf222ae746d8656d156edf13ed"} Sep 10 02:52:00.566395 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:00.566230 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527","Type":"ContainerDied","Data":"eeb15fce8763984da6116a900bbd07f7809e344dceb4161b83c69194673a78a7"} Sep 10 02:52:00.566395 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:00.566270 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527","Type":"ContainerDied","Data":"c2c484e5e7c0f002c0fc991b9f0e09855de7b861aad77e920c0789bebb0a548c"} Sep 10 02:52:01.297774 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.297747 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.331840 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.331802 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-thanos-prometheus-http-client-file\") pod \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " Sep 10 02:52:01.331840 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.331841 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-configmap-kubelet-serving-ca-bundle\") pod \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " Sep 10 02:52:01.332044 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.331870 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-prometheus-k8s-rulefiles-1\") pod \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " Sep 10 02:52:01.332044 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.331902 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-web-config\") pod \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " Sep 10 02:52:01.332044 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.332016 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-prometheus-k8s-rulefiles-2\") pod \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " Sep 10 02:52:01.332191 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.332084 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-prometheus-trusted-ca-bundle\") pod \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " Sep 10 02:52:01.332191 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.332119 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-config\") pod \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " Sep 10 02:52:01.332191 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.332143 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-tls-assets\") pod \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " Sep 10 02:52:01.332191 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.332170 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-config-out\") pod \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " Sep 10 02:52:01.332380 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.332197 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-secret-metrics-client-certs\") pod \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " Sep 10 02:52:01.332380 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.332229 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-secret-prometheus-k8s-tls\") pod \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " Sep 10 02:52:01.332380 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.332259 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-configmap-serving-certs-ca-bundle\") pod \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " Sep 10 02:52:01.332380 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.332277 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" (UID: "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 02:52:01.332380 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.332283 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " Sep 10 02:52:01.332380 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.332350 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5mjd7\" (UniqueName: \"kubernetes.io/projected/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-kube-api-access-5mjd7\") pod \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " Sep 10 02:52:01.332380 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.332310 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" (UID: "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 02:52:01.332795 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.332383 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " Sep 10 02:52:01.332795 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.332397 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" (UID: "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 02:52:01.332795 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.332432 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-prometheus-k8s-rulefiles-0\") pod \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " Sep 10 02:52:01.332795 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.332458 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-configmap-metrics-client-ca\") pod \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " Sep 10 02:52:01.332795 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.332489 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-prometheus-k8s-db\") pod \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " Sep 10 02:52:01.332795 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.332556 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-secret-kube-rbac-proxy\") pod \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " Sep 10 02:52:01.332795 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.332592 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-secret-grpc-tls\") pod \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\" (UID: \"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527\") " Sep 10 02:52:01.333364 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.332862 2560 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-133-37.ec2.internal\" DevicePath \"\"" Sep 10 02:52:01.333364 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.332882 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-133-37.ec2.internal\" DevicePath \"\"" Sep 10 02:52:01.333364 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.332897 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-133-37.ec2.internal\" DevicePath \"\"" Sep 10 02:52:01.335194 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.332484 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" (UID: "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 02:52:01.335194 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.333601 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" (UID: "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 02:52:01.335471 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.335442 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" (UID: "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 02:52:01.335701 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.335656 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" (UID: "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 02:52:01.336763 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.336737 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" (UID: "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 02:52:01.337133 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.337022 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" (UID: "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 02:52:01.337133 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.337073 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" (UID: "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 02:52:01.337133 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.337102 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-kube-api-access-5mjd7" (OuterVolumeSpecName: "kube-api-access-5mjd7") pod "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" (UID: "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527"). InnerVolumeSpecName "kube-api-access-5mjd7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 02:52:01.337133 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.337102 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-config-out" (OuterVolumeSpecName: "config-out") pod "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" (UID: "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 02:52:01.337735 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.337706 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" (UID: "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 02:52:01.337735 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.337712 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" (UID: "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 02:52:01.337921 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.337801 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" (UID: "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 02:52:01.337921 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.337858 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" (UID: "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 02:52:01.338121 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.338092 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" (UID: "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 02:52:01.338987 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.338960 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" (UID: "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 02:52:01.339583 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.339566 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-config" (OuterVolumeSpecName: "config") pod "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" (UID: "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 02:52:01.348439 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.348413 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-web-config" (OuterVolumeSpecName: "web-config") pod "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" (UID: "c9af6a78-5e1f-4f97-9426-9ca4ad6d4527"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 02:52:01.434086 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.434012 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-prometheus-trusted-ca-bundle\") on node \"ip-10-0-133-37.ec2.internal\" DevicePath \"\"" Sep 10 02:52:01.434086 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.434039 2560 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-config\") on node \"ip-10-0-133-37.ec2.internal\" DevicePath \"\"" Sep 10 02:52:01.434086 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.434049 2560 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-tls-assets\") on node \"ip-10-0-133-37.ec2.internal\" DevicePath \"\"" Sep 10 02:52:01.434086 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.434061 2560 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-config-out\") on node \"ip-10-0-133-37.ec2.internal\" DevicePath \"\"" Sep 10 02:52:01.434086 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.434070 2560 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-secret-metrics-client-certs\") on node \"ip-10-0-133-37.ec2.internal\" DevicePath \"\"" Sep 10 02:52:01.434086 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.434079 2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-secret-prometheus-k8s-tls\") on node \"ip-10-0-133-37.ec2.internal\" DevicePath \"\"" Sep 10 02:52:01.434086 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.434088 2560 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-133-37.ec2.internal\" DevicePath \"\"" Sep 10 02:52:01.434384 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.434097 2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-133-37.ec2.internal\" DevicePath \"\"" Sep 10 02:52:01.434384 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.434106 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mjd7\" (UniqueName: \"kubernetes.io/projected/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-kube-api-access-5mjd7\") on node \"ip-10-0-133-37.ec2.internal\" DevicePath \"\"" Sep 10 02:52:01.434384 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.434114 2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-133-37.ec2.internal\" DevicePath \"\"" Sep 10 02:52:01.434384 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.434122 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-133-37.ec2.internal\" DevicePath \"\"" Sep 10 02:52:01.434384 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.434137 2560 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-configmap-metrics-client-ca\") on node \"ip-10-0-133-37.ec2.internal\" DevicePath \"\"" Sep 10 02:52:01.434384 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.434146 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-prometheus-k8s-db\") on node \"ip-10-0-133-37.ec2.internal\" DevicePath \"\"" Sep 10 02:52:01.434384 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.434154 2560 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-secret-kube-rbac-proxy\") on node \"ip-10-0-133-37.ec2.internal\" DevicePath \"\"" Sep 10 02:52:01.434384 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.434162 2560 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-secret-grpc-tls\") on node \"ip-10-0-133-37.ec2.internal\" DevicePath \"\"" Sep 10 02:52:01.434384 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.434170 2560 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-thanos-prometheus-http-client-file\") on node \"ip-10-0-133-37.ec2.internal\" DevicePath \"\"" Sep 10 02:52:01.434384 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.434179 2560 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527-web-config\") on node \"ip-10-0-133-37.ec2.internal\" DevicePath \"\"" Sep 10 02:52:01.573095 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.573057 2560 generic.go:356] "Generic (PLEG): container finished" podID="c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" containerID="a9af9d98465b2ce5dafdae2f51bb0e79c85035520505457bf0bf4a20832d1654" exitCode=0 Sep 10 02:52:01.573266 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.573116 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527","Type":"ContainerDied","Data":"a9af9d98465b2ce5dafdae2f51bb0e79c85035520505457bf0bf4a20832d1654"} Sep 10 02:52:01.573266 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.573132 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.573266 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.573190 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c9af6a78-5e1f-4f97-9426-9ca4ad6d4527","Type":"ContainerDied","Data":"cd9c174415d550c32f8ad568299b3575ae8e266d5f1c9cd75da2814c85184b48"} Sep 10 02:52:01.573266 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.573225 2560 scope.go:117] "RemoveContainer" containerID="fdda688ef50676fd70c570e0f5f7326ddac84e603e94f0eace0bbb56139d4872" Sep 10 02:52:01.581753 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.581737 2560 scope.go:117] "RemoveContainer" containerID="d5c8cde0f1d079a30d41f80c3aa0b0d462b03125fd4ea374dd2e2340e7beb70e" Sep 10 02:52:01.589109 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.589093 2560 scope.go:117] "RemoveContainer" containerID="a9af9d98465b2ce5dafdae2f51bb0e79c85035520505457bf0bf4a20832d1654" Sep 10 02:52:01.596218 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.596200 2560 scope.go:117] "RemoveContainer" containerID="1268318acf5021553b7bcb34cd0dff82b770e3cf222ae746d8656d156edf13ed" Sep 10 02:52:01.603775 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.603759 2560 scope.go:117] "RemoveContainer" containerID="eeb15fce8763984da6116a900bbd07f7809e344dceb4161b83c69194673a78a7" Sep 10 02:52:01.612275 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.612249 2560 scope.go:117] "RemoveContainer" containerID="c2c484e5e7c0f002c0fc991b9f0e09855de7b861aad77e920c0789bebb0a548c" Sep 10 02:52:01.613918 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.613897 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 02:52:01.617703 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.617680 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 02:52:01.621319 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.621304 2560 scope.go:117] "RemoveContainer" containerID="e098823bc59cae2a114b522b7183fed3a9a50ab6a72f6665eeb10ba156d96d1a" Sep 10 02:52:01.628129 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.628113 2560 scope.go:117] "RemoveContainer" containerID="fdda688ef50676fd70c570e0f5f7326ddac84e603e94f0eace0bbb56139d4872" Sep 10 02:52:01.628406 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:52:01.628383 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdda688ef50676fd70c570e0f5f7326ddac84e603e94f0eace0bbb56139d4872\": container with ID starting with fdda688ef50676fd70c570e0f5f7326ddac84e603e94f0eace0bbb56139d4872 not found: ID does not exist" containerID="fdda688ef50676fd70c570e0f5f7326ddac84e603e94f0eace0bbb56139d4872" Sep 10 02:52:01.628468 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.628414 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdda688ef50676fd70c570e0f5f7326ddac84e603e94f0eace0bbb56139d4872"} err="failed to get container status \"fdda688ef50676fd70c570e0f5f7326ddac84e603e94f0eace0bbb56139d4872\": rpc error: code = NotFound desc = could not find container \"fdda688ef50676fd70c570e0f5f7326ddac84e603e94f0eace0bbb56139d4872\": container with ID starting with fdda688ef50676fd70c570e0f5f7326ddac84e603e94f0eace0bbb56139d4872 not found: ID does not exist" Sep 10 02:52:01.628468 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.628447 2560 scope.go:117] "RemoveContainer" containerID="d5c8cde0f1d079a30d41f80c3aa0b0d462b03125fd4ea374dd2e2340e7beb70e" Sep 10 02:52:01.628704 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:52:01.628686 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5c8cde0f1d079a30d41f80c3aa0b0d462b03125fd4ea374dd2e2340e7beb70e\": container with ID starting with d5c8cde0f1d079a30d41f80c3aa0b0d462b03125fd4ea374dd2e2340e7beb70e not found: ID does not exist" containerID="d5c8cde0f1d079a30d41f80c3aa0b0d462b03125fd4ea374dd2e2340e7beb70e" Sep 10 02:52:01.628778 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.628710 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5c8cde0f1d079a30d41f80c3aa0b0d462b03125fd4ea374dd2e2340e7beb70e"} err="failed to get container status \"d5c8cde0f1d079a30d41f80c3aa0b0d462b03125fd4ea374dd2e2340e7beb70e\": rpc error: code = NotFound desc = could not find container \"d5c8cde0f1d079a30d41f80c3aa0b0d462b03125fd4ea374dd2e2340e7beb70e\": container with ID starting with d5c8cde0f1d079a30d41f80c3aa0b0d462b03125fd4ea374dd2e2340e7beb70e not found: ID does not exist" Sep 10 02:52:01.628778 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.628728 2560 scope.go:117] "RemoveContainer" containerID="a9af9d98465b2ce5dafdae2f51bb0e79c85035520505457bf0bf4a20832d1654" Sep 10 02:52:01.628955 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:52:01.628937 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9af9d98465b2ce5dafdae2f51bb0e79c85035520505457bf0bf4a20832d1654\": container with ID starting with a9af9d98465b2ce5dafdae2f51bb0e79c85035520505457bf0bf4a20832d1654 not found: ID does not exist" containerID="a9af9d98465b2ce5dafdae2f51bb0e79c85035520505457bf0bf4a20832d1654" Sep 10 02:52:01.628999 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.628960 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9af9d98465b2ce5dafdae2f51bb0e79c85035520505457bf0bf4a20832d1654"} err="failed to get container status \"a9af9d98465b2ce5dafdae2f51bb0e79c85035520505457bf0bf4a20832d1654\": rpc error: code = NotFound desc = could not find container \"a9af9d98465b2ce5dafdae2f51bb0e79c85035520505457bf0bf4a20832d1654\": container with ID starting with a9af9d98465b2ce5dafdae2f51bb0e79c85035520505457bf0bf4a20832d1654 not found: ID does not exist" Sep 10 02:52:01.628999 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.628972 2560 scope.go:117] "RemoveContainer" containerID="1268318acf5021553b7bcb34cd0dff82b770e3cf222ae746d8656d156edf13ed" Sep 10 02:52:01.629169 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:52:01.629154 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1268318acf5021553b7bcb34cd0dff82b770e3cf222ae746d8656d156edf13ed\": container with ID starting with 1268318acf5021553b7bcb34cd0dff82b770e3cf222ae746d8656d156edf13ed not found: ID does not exist" containerID="1268318acf5021553b7bcb34cd0dff82b770e3cf222ae746d8656d156edf13ed" Sep 10 02:52:01.629208 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.629172 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1268318acf5021553b7bcb34cd0dff82b770e3cf222ae746d8656d156edf13ed"} err="failed to get container status \"1268318acf5021553b7bcb34cd0dff82b770e3cf222ae746d8656d156edf13ed\": rpc error: code = NotFound desc = could not find container \"1268318acf5021553b7bcb34cd0dff82b770e3cf222ae746d8656d156edf13ed\": container with ID starting with 1268318acf5021553b7bcb34cd0dff82b770e3cf222ae746d8656d156edf13ed not found: ID does not exist" Sep 10 02:52:01.629208 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.629184 2560 scope.go:117] "RemoveContainer" containerID="eeb15fce8763984da6116a900bbd07f7809e344dceb4161b83c69194673a78a7" Sep 10 02:52:01.629411 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:52:01.629383 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eeb15fce8763984da6116a900bbd07f7809e344dceb4161b83c69194673a78a7\": container with ID starting with eeb15fce8763984da6116a900bbd07f7809e344dceb4161b83c69194673a78a7 not found: ID does not exist" containerID="eeb15fce8763984da6116a900bbd07f7809e344dceb4161b83c69194673a78a7" Sep 10 02:52:01.629460 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.629419 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eeb15fce8763984da6116a900bbd07f7809e344dceb4161b83c69194673a78a7"} err="failed to get container status \"eeb15fce8763984da6116a900bbd07f7809e344dceb4161b83c69194673a78a7\": rpc error: code = NotFound desc = could not find container \"eeb15fce8763984da6116a900bbd07f7809e344dceb4161b83c69194673a78a7\": container with ID starting with eeb15fce8763984da6116a900bbd07f7809e344dceb4161b83c69194673a78a7 not found: ID does not exist" Sep 10 02:52:01.629460 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.629434 2560 scope.go:117] "RemoveContainer" containerID="c2c484e5e7c0f002c0fc991b9f0e09855de7b861aad77e920c0789bebb0a548c" Sep 10 02:52:01.629726 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:52:01.629710 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2c484e5e7c0f002c0fc991b9f0e09855de7b861aad77e920c0789bebb0a548c\": container with ID starting with c2c484e5e7c0f002c0fc991b9f0e09855de7b861aad77e920c0789bebb0a548c not found: ID does not exist" containerID="c2c484e5e7c0f002c0fc991b9f0e09855de7b861aad77e920c0789bebb0a548c" Sep 10 02:52:01.629771 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.629728 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2c484e5e7c0f002c0fc991b9f0e09855de7b861aad77e920c0789bebb0a548c"} err="failed to get container status \"c2c484e5e7c0f002c0fc991b9f0e09855de7b861aad77e920c0789bebb0a548c\": rpc error: code = NotFound desc = could not find container \"c2c484e5e7c0f002c0fc991b9f0e09855de7b861aad77e920c0789bebb0a548c\": container with ID starting with c2c484e5e7c0f002c0fc991b9f0e09855de7b861aad77e920c0789bebb0a548c not found: ID does not exist" Sep 10 02:52:01.629771 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.629740 2560 scope.go:117] "RemoveContainer" containerID="e098823bc59cae2a114b522b7183fed3a9a50ab6a72f6665eeb10ba156d96d1a" Sep 10 02:52:01.629983 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:52:01.629966 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e098823bc59cae2a114b522b7183fed3a9a50ab6a72f6665eeb10ba156d96d1a\": container with ID starting with e098823bc59cae2a114b522b7183fed3a9a50ab6a72f6665eeb10ba156d96d1a not found: ID does not exist" containerID="e098823bc59cae2a114b522b7183fed3a9a50ab6a72f6665eeb10ba156d96d1a" Sep 10 02:52:01.630042 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.629991 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e098823bc59cae2a114b522b7183fed3a9a50ab6a72f6665eeb10ba156d96d1a"} err="failed to get container status \"e098823bc59cae2a114b522b7183fed3a9a50ab6a72f6665eeb10ba156d96d1a\": rpc error: code = NotFound desc = could not find container \"e098823bc59cae2a114b522b7183fed3a9a50ab6a72f6665eeb10ba156d96d1a\": container with ID starting with e098823bc59cae2a114b522b7183fed3a9a50ab6a72f6665eeb10ba156d96d1a not found: ID does not exist" Sep 10 02:52:01.645812 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.645788 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" path="/var/lib/kubelet/pods/c9af6a78-5e1f-4f97-9426-9ca4ad6d4527/volumes" Sep 10 02:52:01.669183 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.669160 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 02:52:01.669472 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.669458 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4d56e77-0fdc-4733-b06f-2451d59f8343" containerName="registry" Sep 10 02:52:01.669472 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.669472 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4d56e77-0fdc-4733-b06f-2451d59f8343" containerName="registry" Sep 10 02:52:01.669621 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.669482 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" containerName="prometheus" Sep 10 02:52:01.669621 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.669489 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" containerName="prometheus" Sep 10 02:52:01.669621 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.669496 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" containerName="thanos-sidecar" Sep 10 02:52:01.669621 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.669501 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" containerName="thanos-sidecar" Sep 10 02:52:01.669621 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.669535 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" containerName="init-config-reloader" Sep 10 02:52:01.669621 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.669541 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" containerName="init-config-reloader" Sep 10 02:52:01.669621 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.669553 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" containerName="config-reloader" Sep 10 02:52:01.669621 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.669558 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" containerName="config-reloader" Sep 10 02:52:01.669621 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.669567 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" containerName="kube-rbac-proxy-web" Sep 10 02:52:01.669621 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.669571 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" containerName="kube-rbac-proxy-web" Sep 10 02:52:01.669621 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.669578 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" containerName="kube-rbac-proxy" Sep 10 02:52:01.669621 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.669583 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" containerName="kube-rbac-proxy" Sep 10 02:52:01.669621 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.669590 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" containerName="kube-rbac-proxy-thanos" Sep 10 02:52:01.669621 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.669595 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" containerName="kube-rbac-proxy-thanos" Sep 10 02:52:01.670306 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.669642 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4d56e77-0fdc-4733-b06f-2451d59f8343" containerName="registry" Sep 10 02:52:01.670306 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.669651 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" containerName="kube-rbac-proxy" Sep 10 02:52:01.670306 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.669657 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" containerName="kube-rbac-proxy-web" Sep 10 02:52:01.670306 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.669664 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" containerName="prometheus" Sep 10 02:52:01.670306 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.669668 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" containerName="config-reloader" Sep 10 02:52:01.670306 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.669675 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" containerName="kube-rbac-proxy-thanos" Sep 10 02:52:01.670306 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.669681 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9af6a78-5e1f-4f97-9426-9ca4ad6d4527" containerName="thanos-sidecar" Sep 10 02:52:01.673778 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.673762 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.680190 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.679988 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 10 02:52:01.680318 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.680288 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 10 02:52:01.680653 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.680466 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 10 02:52:01.680653 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.680475 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 10 02:52:01.680653 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.680533 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 10 02:52:01.680653 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.680542 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 10 02:52:01.680653 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.680595 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 10 02:52:01.680653 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.680626 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-h92momfi5tiu\"" Sep 10 02:52:01.680999 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.680719 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 10 02:52:01.680999 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.680876 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 10 02:52:01.680999 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.680965 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 10 02:52:01.681939 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.681924 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 10 02:52:01.682552 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.682536 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-rx4lc\"" Sep 10 02:52:01.683344 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.683331 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 10 02:52:01.684398 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.684357 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 10 02:52:01.689439 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.689338 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 10 02:52:01.691705 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.691683 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 10 02:52:01.710808 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.710780 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 02:52:01.736351 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.736312 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.736351 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.736350 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.736599 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.736379 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.736599 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.736430 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.736599 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.736470 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.736599 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.736525 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.736599 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.736551 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.736599 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.736570 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-web-config\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.736599 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.736595 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.736889 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.736623 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.736889 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.736650 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-config\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.736889 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.736664 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.736889 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.736684 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-config-out\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.736889 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.736766 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.736889 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.736803 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.736889 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.736831 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.736889 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.736864 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9s9rn\" (UniqueName: \"kubernetes.io/projected/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-kube-api-access-9s9rn\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.737153 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.736905 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.737153 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.736922 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.737153 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.736949 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.838389 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.838354 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9s9rn\" (UniqueName: \"kubernetes.io/projected/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-kube-api-access-9s9rn\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.838598 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.838403 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.838598 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.838419 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.838598 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.838440 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.838598 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.838459 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.838598 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.838475 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.838598 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.838530 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.838598 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.838551 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.838598 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.838574 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.838598 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.838598 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.839013 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.838624 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.839013 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.838649 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-web-config\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.839013 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.838685 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.839013 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.838714 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.839013 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.838743 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-config\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.839013 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.838765 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.839013 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.838794 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-config-out\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.839013 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.838835 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.839013 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.838859 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.839013 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.838883 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.839482 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.839176 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.839482 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.839234 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.839482 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.839236 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.839482 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.839262 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.841023 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.840997 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.842960 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.842138 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.842960 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.842554 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.842960 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.842644 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.842960 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.842925 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-config\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.843226 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.842961 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.843226 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.843158 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.843418 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.843388 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.843941 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.843882 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-config-out\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.844084 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.844061 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.844151 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.844106 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.844151 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.844120 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.844227 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.844180 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.844317 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.844300 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-web-config\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.845273 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.845258 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.848260 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.848243 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9s9rn\" (UniqueName: \"kubernetes.io/projected/297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4-kube-api-access-9s9rn\") pod \"prometheus-k8s-0\" (UID: \"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:01.983406 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:01.983301 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:52:02.278488 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:02.278458 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 02:52:02.279537 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:52:02.279482 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod297e9b2d_4bb2_456f_a1ba_fd95a63ba8a4.slice/crio-7aaa1279ba3ecbbffa94eb7055e45e0e779860082cb4bf62d573a4b6c34a012c WatchSource:0}: Error finding container 7aaa1279ba3ecbbffa94eb7055e45e0e779860082cb4bf62d573a4b6c34a012c: Status 404 returned error can't find the container with id 7aaa1279ba3ecbbffa94eb7055e45e0e779860082cb4bf62d573a4b6c34a012c Sep 10 02:52:02.578961 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:02.578869 2560 generic.go:356] "Generic (PLEG): container finished" podID="297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4" containerID="4c4991287c5e34b45c452fa16519db58a4bc268b660de0fd2d2e0609dfdb5cd0" exitCode=0 Sep 10 02:52:02.579384 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:02.578964 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4","Type":"ContainerDied","Data":"4c4991287c5e34b45c452fa16519db58a4bc268b660de0fd2d2e0609dfdb5cd0"} Sep 10 02:52:02.579384 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:02.579042 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4","Type":"ContainerStarted","Data":"7aaa1279ba3ecbbffa94eb7055e45e0e779860082cb4bf62d573a4b6c34a012c"} Sep 10 02:52:03.587650 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:03.587562 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4","Type":"ContainerStarted","Data":"3c19e794f6c2e4fc4428361cb038ed246428cdff35b7913396da2bfb3b7ba8f4"} Sep 10 02:52:03.587650 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:03.587627 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4","Type":"ContainerStarted","Data":"5c6d6a832030bbc6ecc2d0bc146f4660d364e6e61d9f96fea8468cc7e0c9428a"} Sep 10 02:52:03.588048 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:03.587653 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4","Type":"ContainerStarted","Data":"24dd2fd50d23805d5fb2217d1351b9f805c2f923eb34bb5d37e562d59209dcb7"} Sep 10 02:52:03.588048 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:03.587677 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4","Type":"ContainerStarted","Data":"5bf02fb02c69165df9098fdc1e43b93d16de6c40bb89b35a1a269098a982ac39"} Sep 10 02:52:03.588048 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:03.587701 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4","Type":"ContainerStarted","Data":"5c92d843c64653c6bb137781cdcef7874d731549a6a7e206b61431741ad051ec"} Sep 10 02:52:03.588048 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:03.587724 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4","Type":"ContainerStarted","Data":"e6ef2753ee6af88fdc2496609fe809ca0d506017e5d6fd842506f1fab1aee94f"} Sep 10 02:52:03.617549 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:03.617398 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.598566065 podStartE2EDuration="2.617369503s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.803438ms" isStatefulPod=false podCreationTimestamp="2026-09-10 02:52:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:52:03.616012001 +0000 UTC m=+182.462841268" watchObservedRunningTime="2026-09-10 02:52:03.617369503 +0000 UTC m=+182.464198772" Sep 10 02:52:06.983439 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:52:06.983404 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:53:01.984108 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:53:01.984064 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:53:02.000360 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:53:02.000335 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:53:02.847923 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:53:02.847892 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 02:54:01.523762 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:01.523731 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ghs46_a5210aa8-4171-4cc3-b65b-6237a9aee1d6/console-operator/2.log" Sep 10 02:54:01.524372 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:01.524339 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ghs46_a5210aa8-4171-4cc3-b65b-6237a9aee1d6/console-operator/2.log" Sep 10 02:54:01.537829 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:01.537797 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swhbm_8a6680c4-21b8-4fe2-a763-e84858c7ef90/ovn-acl-logging/0.log" Sep 10 02:54:01.538790 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:01.538768 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swhbm_8a6680c4-21b8-4fe2-a763-e84858c7ef90/ovn-acl-logging/0.log" Sep 10 02:54:01.544227 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:01.544208 2560 kubelet.go:1648] "Image garbage collection succeeded" Sep 10 02:54:01.628384 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:01.628351 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-79bc984b7c-nc22k"] Sep 10 02:54:01.631070 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:01.631054 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-79bc984b7c-nc22k" Sep 10 02:54:01.634025 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:01.633999 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Sep 10 02:54:01.634913 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:01.634889 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Sep 10 02:54:01.635690 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:01.635669 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Sep 10 02:54:01.636469 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:01.636452 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-t6jkp\"" Sep 10 02:54:01.636581 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:01.636500 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Sep 10 02:54:01.636994 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:01.636975 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Sep 10 02:54:01.654080 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:01.654058 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-79bc984b7c-nc22k"] Sep 10 02:54:01.782384 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:01.782290 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e1791a46-f213-4ad8-abc1-3358cb12b25d-metrics-cert\") pod \"lws-controller-manager-79bc984b7c-nc22k\" (UID: \"e1791a46-f213-4ad8-abc1-3358cb12b25d\") " pod="openshift-lws-operator/lws-controller-manager-79bc984b7c-nc22k" Sep 10 02:54:01.782384 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:01.782333 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e1791a46-f213-4ad8-abc1-3358cb12b25d-manager-config\") pod \"lws-controller-manager-79bc984b7c-nc22k\" (UID: \"e1791a46-f213-4ad8-abc1-3358cb12b25d\") " pod="openshift-lws-operator/lws-controller-manager-79bc984b7c-nc22k" Sep 10 02:54:01.782629 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:01.782389 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e1791a46-f213-4ad8-abc1-3358cb12b25d-cert\") pod \"lws-controller-manager-79bc984b7c-nc22k\" (UID: \"e1791a46-f213-4ad8-abc1-3358cb12b25d\") " pod="openshift-lws-operator/lws-controller-manager-79bc984b7c-nc22k" Sep 10 02:54:01.782629 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:01.782452 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xb4tl\" (UniqueName: \"kubernetes.io/projected/e1791a46-f213-4ad8-abc1-3358cb12b25d-kube-api-access-xb4tl\") pod \"lws-controller-manager-79bc984b7c-nc22k\" (UID: \"e1791a46-f213-4ad8-abc1-3358cb12b25d\") " pod="openshift-lws-operator/lws-controller-manager-79bc984b7c-nc22k" Sep 10 02:54:01.883850 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:01.883816 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xb4tl\" (UniqueName: \"kubernetes.io/projected/e1791a46-f213-4ad8-abc1-3358cb12b25d-kube-api-access-xb4tl\") pod \"lws-controller-manager-79bc984b7c-nc22k\" (UID: \"e1791a46-f213-4ad8-abc1-3358cb12b25d\") " pod="openshift-lws-operator/lws-controller-manager-79bc984b7c-nc22k" Sep 10 02:54:01.884037 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:01.883876 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e1791a46-f213-4ad8-abc1-3358cb12b25d-metrics-cert\") pod \"lws-controller-manager-79bc984b7c-nc22k\" (UID: \"e1791a46-f213-4ad8-abc1-3358cb12b25d\") " pod="openshift-lws-operator/lws-controller-manager-79bc984b7c-nc22k" Sep 10 02:54:01.884037 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:01.883896 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e1791a46-f213-4ad8-abc1-3358cb12b25d-manager-config\") pod \"lws-controller-manager-79bc984b7c-nc22k\" (UID: \"e1791a46-f213-4ad8-abc1-3358cb12b25d\") " pod="openshift-lws-operator/lws-controller-manager-79bc984b7c-nc22k" Sep 10 02:54:01.884037 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:01.883954 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e1791a46-f213-4ad8-abc1-3358cb12b25d-cert\") pod \"lws-controller-manager-79bc984b7c-nc22k\" (UID: \"e1791a46-f213-4ad8-abc1-3358cb12b25d\") " pod="openshift-lws-operator/lws-controller-manager-79bc984b7c-nc22k" Sep 10 02:54:01.884646 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:01.884623 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e1791a46-f213-4ad8-abc1-3358cb12b25d-manager-config\") pod \"lws-controller-manager-79bc984b7c-nc22k\" (UID: \"e1791a46-f213-4ad8-abc1-3358cb12b25d\") " pod="openshift-lws-operator/lws-controller-manager-79bc984b7c-nc22k" Sep 10 02:54:01.886520 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:01.886481 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e1791a46-f213-4ad8-abc1-3358cb12b25d-metrics-cert\") pod \"lws-controller-manager-79bc984b7c-nc22k\" (UID: \"e1791a46-f213-4ad8-abc1-3358cb12b25d\") " pod="openshift-lws-operator/lws-controller-manager-79bc984b7c-nc22k" Sep 10 02:54:01.886609 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:01.886559 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e1791a46-f213-4ad8-abc1-3358cb12b25d-cert\") pod \"lws-controller-manager-79bc984b7c-nc22k\" (UID: \"e1791a46-f213-4ad8-abc1-3358cb12b25d\") " pod="openshift-lws-operator/lws-controller-manager-79bc984b7c-nc22k" Sep 10 02:54:01.893565 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:01.893542 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xb4tl\" (UniqueName: \"kubernetes.io/projected/e1791a46-f213-4ad8-abc1-3358cb12b25d-kube-api-access-xb4tl\") pod \"lws-controller-manager-79bc984b7c-nc22k\" (UID: \"e1791a46-f213-4ad8-abc1-3358cb12b25d\") " pod="openshift-lws-operator/lws-controller-manager-79bc984b7c-nc22k" Sep 10 02:54:01.942100 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:01.942052 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-79bc984b7c-nc22k" Sep 10 02:54:02.230114 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:02.230085 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-79bc984b7c-nc22k"] Sep 10 02:54:02.232532 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:54:02.232478 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1791a46_f213_4ad8_abc1_3358cb12b25d.slice/crio-6562e7f0e0548e61d4f0d41ffc657780c602ac57964c3d1b52a08257c3d8401a WatchSource:0}: Error finding container 6562e7f0e0548e61d4f0d41ffc657780c602ac57964c3d1b52a08257c3d8401a: Status 404 returned error can't find the container with id 6562e7f0e0548e61d4f0d41ffc657780c602ac57964c3d1b52a08257c3d8401a Sep 10 02:54:03.082901 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:03.082855 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-79bc984b7c-nc22k" event={"ID":"e1791a46-f213-4ad8-abc1-3358cb12b25d","Type":"ContainerStarted","Data":"6562e7f0e0548e61d4f0d41ffc657780c602ac57964c3d1b52a08257c3d8401a"} Sep 10 02:54:04.712925 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:04.712893 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78f567c59b-bd7km"] Sep 10 02:54:04.716335 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:04.716319 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-78f567c59b-bd7km" Sep 10 02:54:04.718501 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:04.718481 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 10 02:54:04.718831 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:04.718814 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 10 02:54:04.718901 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:04.718884 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 10 02:54:04.719318 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:04.719298 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7zgks\"" Sep 10 02:54:04.719411 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:04.719373 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 10 02:54:04.735401 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:04.735359 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78f567c59b-bd7km"] Sep 10 02:54:04.812234 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:04.812129 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a18dd49f-0a00-4a30-b4c4-ed8facc0afdc-webhook-cert\") pod \"opendatahub-operator-controller-manager-78f567c59b-bd7km\" (UID: \"a18dd49f-0a00-4a30-b4c4-ed8facc0afdc\") " pod="opendatahub/opendatahub-operator-controller-manager-78f567c59b-bd7km" Sep 10 02:54:04.812234 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:04.812174 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz8gr\" (UniqueName: \"kubernetes.io/projected/a18dd49f-0a00-4a30-b4c4-ed8facc0afdc-kube-api-access-jz8gr\") pod \"opendatahub-operator-controller-manager-78f567c59b-bd7km\" (UID: \"a18dd49f-0a00-4a30-b4c4-ed8facc0afdc\") " pod="opendatahub/opendatahub-operator-controller-manager-78f567c59b-bd7km" Sep 10 02:54:04.812483 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:04.812246 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a18dd49f-0a00-4a30-b4c4-ed8facc0afdc-manifests\") pod \"opendatahub-operator-controller-manager-78f567c59b-bd7km\" (UID: \"a18dd49f-0a00-4a30-b4c4-ed8facc0afdc\") " pod="opendatahub/opendatahub-operator-controller-manager-78f567c59b-bd7km" Sep 10 02:54:04.812483 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:04.812285 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a18dd49f-0a00-4a30-b4c4-ed8facc0afdc-tmp\") pod \"opendatahub-operator-controller-manager-78f567c59b-bd7km\" (UID: \"a18dd49f-0a00-4a30-b4c4-ed8facc0afdc\") " pod="opendatahub/opendatahub-operator-controller-manager-78f567c59b-bd7km" Sep 10 02:54:04.812483 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:04.812346 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a18dd49f-0a00-4a30-b4c4-ed8facc0afdc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78f567c59b-bd7km\" (UID: \"a18dd49f-0a00-4a30-b4c4-ed8facc0afdc\") " pod="opendatahub/opendatahub-operator-controller-manager-78f567c59b-bd7km" Sep 10 02:54:04.913491 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:04.913445 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a18dd49f-0a00-4a30-b4c4-ed8facc0afdc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78f567c59b-bd7km\" (UID: \"a18dd49f-0a00-4a30-b4c4-ed8facc0afdc\") " pod="opendatahub/opendatahub-operator-controller-manager-78f567c59b-bd7km" Sep 10 02:54:04.913729 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:04.913550 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a18dd49f-0a00-4a30-b4c4-ed8facc0afdc-webhook-cert\") pod \"opendatahub-operator-controller-manager-78f567c59b-bd7km\" (UID: \"a18dd49f-0a00-4a30-b4c4-ed8facc0afdc\") " pod="opendatahub/opendatahub-operator-controller-manager-78f567c59b-bd7km" Sep 10 02:54:04.913729 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:04.913587 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz8gr\" (UniqueName: \"kubernetes.io/projected/a18dd49f-0a00-4a30-b4c4-ed8facc0afdc-kube-api-access-jz8gr\") pod \"opendatahub-operator-controller-manager-78f567c59b-bd7km\" (UID: \"a18dd49f-0a00-4a30-b4c4-ed8facc0afdc\") " pod="opendatahub/opendatahub-operator-controller-manager-78f567c59b-bd7km" Sep 10 02:54:04.913841 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:04.913712 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a18dd49f-0a00-4a30-b4c4-ed8facc0afdc-manifests\") pod \"opendatahub-operator-controller-manager-78f567c59b-bd7km\" (UID: \"a18dd49f-0a00-4a30-b4c4-ed8facc0afdc\") " pod="opendatahub/opendatahub-operator-controller-manager-78f567c59b-bd7km" Sep 10 02:54:04.913841 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:04.913779 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a18dd49f-0a00-4a30-b4c4-ed8facc0afdc-tmp\") pod \"opendatahub-operator-controller-manager-78f567c59b-bd7km\" (UID: \"a18dd49f-0a00-4a30-b4c4-ed8facc0afdc\") " pod="opendatahub/opendatahub-operator-controller-manager-78f567c59b-bd7km" Sep 10 02:54:04.914083 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:04.914063 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a18dd49f-0a00-4a30-b4c4-ed8facc0afdc-manifests\") pod \"opendatahub-operator-controller-manager-78f567c59b-bd7km\" (UID: \"a18dd49f-0a00-4a30-b4c4-ed8facc0afdc\") " pod="opendatahub/opendatahub-operator-controller-manager-78f567c59b-bd7km" Sep 10 02:54:04.916025 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:04.916000 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a18dd49f-0a00-4a30-b4c4-ed8facc0afdc-tmp\") pod \"opendatahub-operator-controller-manager-78f567c59b-bd7km\" (UID: \"a18dd49f-0a00-4a30-b4c4-ed8facc0afdc\") " pod="opendatahub/opendatahub-operator-controller-manager-78f567c59b-bd7km" Sep 10 02:54:04.916251 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:04.916233 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a18dd49f-0a00-4a30-b4c4-ed8facc0afdc-webhook-cert\") pod \"opendatahub-operator-controller-manager-78f567c59b-bd7km\" (UID: \"a18dd49f-0a00-4a30-b4c4-ed8facc0afdc\") " pod="opendatahub/opendatahub-operator-controller-manager-78f567c59b-bd7km" Sep 10 02:54:04.916326 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:04.916312 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a18dd49f-0a00-4a30-b4c4-ed8facc0afdc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78f567c59b-bd7km\" (UID: \"a18dd49f-0a00-4a30-b4c4-ed8facc0afdc\") " pod="opendatahub/opendatahub-operator-controller-manager-78f567c59b-bd7km" Sep 10 02:54:04.921495 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:04.921470 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz8gr\" (UniqueName: \"kubernetes.io/projected/a18dd49f-0a00-4a30-b4c4-ed8facc0afdc-kube-api-access-jz8gr\") pod \"opendatahub-operator-controller-manager-78f567c59b-bd7km\" (UID: \"a18dd49f-0a00-4a30-b4c4-ed8facc0afdc\") " pod="opendatahub/opendatahub-operator-controller-manager-78f567c59b-bd7km" Sep 10 02:54:05.026966 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:05.026929 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-78f567c59b-bd7km" Sep 10 02:54:05.095574 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:05.095474 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-79bc984b7c-nc22k" event={"ID":"e1791a46-f213-4ad8-abc1-3358cb12b25d","Type":"ContainerStarted","Data":"8e9ec811736d34303175c64238ec1affd09fd587a9125c5c3a8a79b7c0436ed2"} Sep 10 02:54:05.096371 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:05.096346 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-79bc984b7c-nc22k" Sep 10 02:54:05.126976 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:05.126922 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-79bc984b7c-nc22k" podStartSLOduration=1.8631539830000001 podStartE2EDuration="4.126903619s" totalImagesPullingTime="2.263749636s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 02:54:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:54:05.12501788 +0000 UTC m=+303.971847147" watchObservedRunningTime="2026-09-10 02:54:05.126903619 +0000 UTC m=+303.973732886" Sep 10 02:54:05.304767 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:05.304744 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78f567c59b-bd7km"] Sep 10 02:54:05.308369 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:54:05.308336 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda18dd49f_0a00_4a30_b4c4_ed8facc0afdc.slice/crio-34477b6aaf3bfe2ca55fcc22fdaac721e3c0a8fe301062ad756ae77951de5d3c WatchSource:0}: Error finding container 34477b6aaf3bfe2ca55fcc22fdaac721e3c0a8fe301062ad756ae77951de5d3c: Status 404 returned error can't find the container with id 34477b6aaf3bfe2ca55fcc22fdaac721e3c0a8fe301062ad756ae77951de5d3c Sep 10 02:54:05.309543 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:05.309480 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 02:54:06.103781 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:06.103740 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78f567c59b-bd7km" event={"ID":"a18dd49f-0a00-4a30-b4c4-ed8facc0afdc","Type":"ContainerStarted","Data":"34477b6aaf3bfe2ca55fcc22fdaac721e3c0a8fe301062ad756ae77951de5d3c"} Sep 10 02:54:09.120234 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:09.120198 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78f567c59b-bd7km" event={"ID":"a18dd49f-0a00-4a30-b4c4-ed8facc0afdc","Type":"ContainerStarted","Data":"1bc7252bf91ae1f3be607eaf18fb06de2aba3a77b1f60bc781a8712baad3ee3e"} Sep 10 02:54:10.129436 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:10.129404 2560 generic.go:356] "Generic (PLEG): container finished" podID="a18dd49f-0a00-4a30-b4c4-ed8facc0afdc" containerID="1bc7252bf91ae1f3be607eaf18fb06de2aba3a77b1f60bc781a8712baad3ee3e" exitCode=0 Sep 10 02:54:10.129859 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:10.129449 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78f567c59b-bd7km" event={"ID":"a18dd49f-0a00-4a30-b4c4-ed8facc0afdc","Type":"ContainerDied","Data":"1bc7252bf91ae1f3be607eaf18fb06de2aba3a77b1f60bc781a8712baad3ee3e"} Sep 10 02:54:11.135582 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:11.135544 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78f567c59b-bd7km" event={"ID":"a18dd49f-0a00-4a30-b4c4-ed8facc0afdc","Type":"ContainerStarted","Data":"8420b7c14f740d0132dd6761fd48454a7210fc9e4aa5554ff9182dc8026a5d09"} Sep 10 02:54:11.135961 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:11.135627 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-78f567c59b-bd7km" Sep 10 02:54:11.156205 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:11.156142 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-78f567c59b-bd7km" podStartSLOduration=2.692077198 podStartE2EDuration="7.156122161s" totalImagesPullingTime="3.385968312s" totalInitContainerRuntime="1.078076651s" isStatefulPod=false podCreationTimestamp="2026-09-10 02:54:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:54:11.155980814 +0000 UTC m=+310.002810093" watchObservedRunningTime="2026-09-10 02:54:11.156122161 +0000 UTC m=+310.002951430" Sep 10 02:54:17.111959 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:17.111926 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-79bc984b7c-nc22k" Sep 10 02:54:22.143325 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:22.143292 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-78f567c59b-bd7km" Sep 10 02:54:43.396480 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.396447 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c"] Sep 10 02:54:43.403413 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.403388 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c" Sep 10 02:54:43.406041 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.406012 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-z6l9x\"" Sep 10 02:54:43.406041 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.406014 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 10 02:54:43.411078 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.411049 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c"] Sep 10 02:54:43.507014 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.506973 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d25f3ad4-85b1-4af2-b985-f2540349cda0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c\" (UID: \"d25f3ad4-85b1-4af2-b985-f2540349cda0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c" Sep 10 02:54:43.507192 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.507039 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d25f3ad4-85b1-4af2-b985-f2540349cda0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c\" (UID: \"d25f3ad4-85b1-4af2-b985-f2540349cda0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c" Sep 10 02:54:43.507192 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.507060 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d25f3ad4-85b1-4af2-b985-f2540349cda0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c\" (UID: \"d25f3ad4-85b1-4af2-b985-f2540349cda0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c" Sep 10 02:54:43.507192 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.507076 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d25f3ad4-85b1-4af2-b985-f2540349cda0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c\" (UID: \"d25f3ad4-85b1-4af2-b985-f2540349cda0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c" Sep 10 02:54:43.507192 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.507092 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d25f3ad4-85b1-4af2-b985-f2540349cda0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c\" (UID: \"d25f3ad4-85b1-4af2-b985-f2540349cda0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c" Sep 10 02:54:43.507192 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.507111 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d25f3ad4-85b1-4af2-b985-f2540349cda0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c\" (UID: \"d25f3ad4-85b1-4af2-b985-f2540349cda0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c" Sep 10 02:54:43.507192 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.507128 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d25f3ad4-85b1-4af2-b985-f2540349cda0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c\" (UID: \"d25f3ad4-85b1-4af2-b985-f2540349cda0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c" Sep 10 02:54:43.507384 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.507193 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d25f3ad4-85b1-4af2-b985-f2540349cda0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c\" (UID: \"d25f3ad4-85b1-4af2-b985-f2540349cda0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c" Sep 10 02:54:43.507384 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.507241 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwwwh\" (UniqueName: \"kubernetes.io/projected/d25f3ad4-85b1-4af2-b985-f2540349cda0-kube-api-access-xwwwh\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c\" (UID: \"d25f3ad4-85b1-4af2-b985-f2540349cda0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c" Sep 10 02:54:43.608336 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.608301 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d25f3ad4-85b1-4af2-b985-f2540349cda0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c\" (UID: \"d25f3ad4-85b1-4af2-b985-f2540349cda0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c" Sep 10 02:54:43.608336 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.608344 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d25f3ad4-85b1-4af2-b985-f2540349cda0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c\" (UID: \"d25f3ad4-85b1-4af2-b985-f2540349cda0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c" Sep 10 02:54:43.608628 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.608361 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d25f3ad4-85b1-4af2-b985-f2540349cda0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c\" (UID: \"d25f3ad4-85b1-4af2-b985-f2540349cda0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c" Sep 10 02:54:43.608628 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.608378 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d25f3ad4-85b1-4af2-b985-f2540349cda0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c\" (UID: \"d25f3ad4-85b1-4af2-b985-f2540349cda0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c" Sep 10 02:54:43.608628 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.608397 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d25f3ad4-85b1-4af2-b985-f2540349cda0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c\" (UID: \"d25f3ad4-85b1-4af2-b985-f2540349cda0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c" Sep 10 02:54:43.608628 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.608550 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d25f3ad4-85b1-4af2-b985-f2540349cda0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c\" (UID: \"d25f3ad4-85b1-4af2-b985-f2540349cda0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c" Sep 10 02:54:43.608628 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.608596 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d25f3ad4-85b1-4af2-b985-f2540349cda0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c\" (UID: \"d25f3ad4-85b1-4af2-b985-f2540349cda0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c" Sep 10 02:54:43.608955 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.608644 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwwwh\" (UniqueName: \"kubernetes.io/projected/d25f3ad4-85b1-4af2-b985-f2540349cda0-kube-api-access-xwwwh\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c\" (UID: \"d25f3ad4-85b1-4af2-b985-f2540349cda0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c" Sep 10 02:54:43.608955 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.608689 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d25f3ad4-85b1-4af2-b985-f2540349cda0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c\" (UID: \"d25f3ad4-85b1-4af2-b985-f2540349cda0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c" Sep 10 02:54:43.608955 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.608750 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d25f3ad4-85b1-4af2-b985-f2540349cda0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c\" (UID: \"d25f3ad4-85b1-4af2-b985-f2540349cda0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c" Sep 10 02:54:43.608955 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.608853 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d25f3ad4-85b1-4af2-b985-f2540349cda0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c\" (UID: \"d25f3ad4-85b1-4af2-b985-f2540349cda0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c" Sep 10 02:54:43.609246 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.609001 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d25f3ad4-85b1-4af2-b985-f2540349cda0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c\" (UID: \"d25f3ad4-85b1-4af2-b985-f2540349cda0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c" Sep 10 02:54:43.609246 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.609036 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d25f3ad4-85b1-4af2-b985-f2540349cda0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c\" (UID: \"d25f3ad4-85b1-4af2-b985-f2540349cda0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c" Sep 10 02:54:43.609427 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.609360 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d25f3ad4-85b1-4af2-b985-f2540349cda0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c\" (UID: \"d25f3ad4-85b1-4af2-b985-f2540349cda0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c" Sep 10 02:54:43.610912 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.610888 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d25f3ad4-85b1-4af2-b985-f2540349cda0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c\" (UID: \"d25f3ad4-85b1-4af2-b985-f2540349cda0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c" Sep 10 02:54:43.611108 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.611093 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d25f3ad4-85b1-4af2-b985-f2540349cda0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c\" (UID: \"d25f3ad4-85b1-4af2-b985-f2540349cda0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c" Sep 10 02:54:43.616672 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.616653 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d25f3ad4-85b1-4af2-b985-f2540349cda0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c\" (UID: \"d25f3ad4-85b1-4af2-b985-f2540349cda0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c" Sep 10 02:54:43.617244 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.617222 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwwwh\" (UniqueName: \"kubernetes.io/projected/d25f3ad4-85b1-4af2-b985-f2540349cda0-kube-api-access-xwwwh\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c\" (UID: \"d25f3ad4-85b1-4af2-b985-f2540349cda0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c" Sep 10 02:54:43.697801 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.697719 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-rvcrj"] Sep 10 02:54:43.700790 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.700773 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rvcrj" Sep 10 02:54:43.704308 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.704282 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Sep 10 02:54:43.704570 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.704553 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Sep 10 02:54:43.704766 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.704750 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Sep 10 02:54:43.705344 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.705330 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-cxmz5\"" Sep 10 02:54:43.709721 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.709701 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/11a1299c-7c3b-4f2b-8a77-6decfb51dbbb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-rvcrj\" (UID: \"11a1299c-7c3b-4f2b-8a77-6decfb51dbbb\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rvcrj" Sep 10 02:54:43.709833 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.709760 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvwhb\" (UniqueName: \"kubernetes.io/projected/11a1299c-7c3b-4f2b-8a77-6decfb51dbbb-kube-api-access-vvwhb\") pod \"istiod-openshift-gateway-5677ccb49b-rvcrj\" (UID: \"11a1299c-7c3b-4f2b-8a77-6decfb51dbbb\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rvcrj" Sep 10 02:54:43.709897 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.709851 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11a1299c-7c3b-4f2b-8a77-6decfb51dbbb-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-rvcrj\" (UID: \"11a1299c-7c3b-4f2b-8a77-6decfb51dbbb\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rvcrj" Sep 10 02:54:43.709897 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.709877 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/11a1299c-7c3b-4f2b-8a77-6decfb51dbbb-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-rvcrj\" (UID: \"11a1299c-7c3b-4f2b-8a77-6decfb51dbbb\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rvcrj" Sep 10 02:54:43.709999 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.709919 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/11a1299c-7c3b-4f2b-8a77-6decfb51dbbb-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-rvcrj\" (UID: \"11a1299c-7c3b-4f2b-8a77-6decfb51dbbb\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rvcrj" Sep 10 02:54:43.709999 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.709963 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/11a1299c-7c3b-4f2b-8a77-6decfb51dbbb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-rvcrj\" (UID: \"11a1299c-7c3b-4f2b-8a77-6decfb51dbbb\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rvcrj" Sep 10 02:54:43.710094 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.710001 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/11a1299c-7c3b-4f2b-8a77-6decfb51dbbb-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-rvcrj\" (UID: \"11a1299c-7c3b-4f2b-8a77-6decfb51dbbb\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rvcrj" Sep 10 02:54:43.714891 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.714873 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c" Sep 10 02:54:43.725341 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.725322 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-rvcrj"] Sep 10 02:54:43.810979 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.810943 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvwhb\" (UniqueName: \"kubernetes.io/projected/11a1299c-7c3b-4f2b-8a77-6decfb51dbbb-kube-api-access-vvwhb\") pod \"istiod-openshift-gateway-5677ccb49b-rvcrj\" (UID: \"11a1299c-7c3b-4f2b-8a77-6decfb51dbbb\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rvcrj" Sep 10 02:54:43.811173 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.810999 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11a1299c-7c3b-4f2b-8a77-6decfb51dbbb-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-rvcrj\" (UID: \"11a1299c-7c3b-4f2b-8a77-6decfb51dbbb\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rvcrj" Sep 10 02:54:43.811173 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.811017 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/11a1299c-7c3b-4f2b-8a77-6decfb51dbbb-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-rvcrj\" (UID: \"11a1299c-7c3b-4f2b-8a77-6decfb51dbbb\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rvcrj" Sep 10 02:54:43.811173 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.811034 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/11a1299c-7c3b-4f2b-8a77-6decfb51dbbb-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-rvcrj\" (UID: \"11a1299c-7c3b-4f2b-8a77-6decfb51dbbb\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rvcrj" Sep 10 02:54:43.811173 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.811075 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/11a1299c-7c3b-4f2b-8a77-6decfb51dbbb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-rvcrj\" (UID: \"11a1299c-7c3b-4f2b-8a77-6decfb51dbbb\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rvcrj" Sep 10 02:54:43.811173 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.811108 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/11a1299c-7c3b-4f2b-8a77-6decfb51dbbb-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-rvcrj\" (UID: \"11a1299c-7c3b-4f2b-8a77-6decfb51dbbb\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rvcrj" Sep 10 02:54:43.811173 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.811171 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/11a1299c-7c3b-4f2b-8a77-6decfb51dbbb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-rvcrj\" (UID: \"11a1299c-7c3b-4f2b-8a77-6decfb51dbbb\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rvcrj" Sep 10 02:54:43.812053 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.812016 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/11a1299c-7c3b-4f2b-8a77-6decfb51dbbb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-rvcrj\" (UID: \"11a1299c-7c3b-4f2b-8a77-6decfb51dbbb\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rvcrj" Sep 10 02:54:43.813769 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.813744 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/11a1299c-7c3b-4f2b-8a77-6decfb51dbbb-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-rvcrj\" (UID: \"11a1299c-7c3b-4f2b-8a77-6decfb51dbbb\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rvcrj" Sep 10 02:54:43.813878 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.813746 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/11a1299c-7c3b-4f2b-8a77-6decfb51dbbb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-rvcrj\" (UID: \"11a1299c-7c3b-4f2b-8a77-6decfb51dbbb\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rvcrj" Sep 10 02:54:43.813878 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.813793 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/11a1299c-7c3b-4f2b-8a77-6decfb51dbbb-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-rvcrj\" (UID: \"11a1299c-7c3b-4f2b-8a77-6decfb51dbbb\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rvcrj" Sep 10 02:54:43.813878 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.813865 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/11a1299c-7c3b-4f2b-8a77-6decfb51dbbb-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-rvcrj\" (UID: \"11a1299c-7c3b-4f2b-8a77-6decfb51dbbb\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rvcrj" Sep 10 02:54:43.831444 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.831399 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvwhb\" (UniqueName: \"kubernetes.io/projected/11a1299c-7c3b-4f2b-8a77-6decfb51dbbb-kube-api-access-vvwhb\") pod \"istiod-openshift-gateway-5677ccb49b-rvcrj\" (UID: \"11a1299c-7c3b-4f2b-8a77-6decfb51dbbb\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rvcrj" Sep 10 02:54:43.833744 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.833724 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11a1299c-7c3b-4f2b-8a77-6decfb51dbbb-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-rvcrj\" (UID: \"11a1299c-7c3b-4f2b-8a77-6decfb51dbbb\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rvcrj" Sep 10 02:54:43.992160 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:43.992071 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c"] Sep 10 02:54:43.996378 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:54:43.996348 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd25f3ad4_85b1_4af2_b985_f2540349cda0.slice/crio-3040d426f13b00435cbe1776b53bfb64f62587a1446afe4b409606b111d75c95 WatchSource:0}: Error finding container 3040d426f13b00435cbe1776b53bfb64f62587a1446afe4b409606b111d75c95: Status 404 returned error can't find the container with id 3040d426f13b00435cbe1776b53bfb64f62587a1446afe4b409606b111d75c95 Sep 10 02:54:44.010670 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:44.010650 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rvcrj" Sep 10 02:54:44.284008 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:44.283922 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c" event={"ID":"d25f3ad4-85b1-4af2-b985-f2540349cda0","Type":"ContainerStarted","Data":"3040d426f13b00435cbe1776b53bfb64f62587a1446afe4b409606b111d75c95"} Sep 10 02:54:44.289324 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:44.289294 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-rvcrj"] Sep 10 02:54:44.292686 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:54:44.292661 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11a1299c_7c3b_4f2b_8a77_6decfb51dbbb.slice/crio-be731d0cc7dbb48564c9a08bb4b95d5fdf9994f274d45f2319a80b20c0c1b972 WatchSource:0}: Error finding container be731d0cc7dbb48564c9a08bb4b95d5fdf9994f274d45f2319a80b20c0c1b972: Status 404 returned error can't find the container with id be731d0cc7dbb48564c9a08bb4b95d5fdf9994f274d45f2319a80b20c0c1b972 Sep 10 02:54:45.291540 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:45.291442 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rvcrj" event={"ID":"11a1299c-7c3b-4f2b-8a77-6decfb51dbbb","Type":"ContainerStarted","Data":"be731d0cc7dbb48564c9a08bb4b95d5fdf9994f274d45f2319a80b20c0c1b972"} Sep 10 02:54:47.687122 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:47.687082 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Sep 10 02:54:47.687415 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:47.687161 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Sep 10 02:54:47.687415 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:47.687191 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Sep 10 02:54:47.689719 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:47.689685 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Sep 10 02:54:47.689799 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:47.689751 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Sep 10 02:54:48.311860 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:48.311781 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rvcrj" event={"ID":"11a1299c-7c3b-4f2b-8a77-6decfb51dbbb","Type":"ContainerStarted","Data":"4a6a9eddc2b042486e3a5fff6d5659167a30ce2db275f3d87e7642814c0de951"} Sep 10 02:54:48.312389 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:48.312347 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rvcrj" Sep 10 02:54:48.314331 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:48.314301 2560 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-rvcrj container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 10 02:54:48.314448 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:48.314360 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rvcrj" podUID="11a1299c-7c3b-4f2b-8a77-6decfb51dbbb" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 10 02:54:48.314448 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:48.314404 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c" event={"ID":"d25f3ad4-85b1-4af2-b985-f2540349cda0","Type":"ContainerStarted","Data":"89b8a1a7c8108d4a0dcf06ec9e4f2535449ffc57b3e1a06673862465cb36fec9"} Sep 10 02:54:48.371907 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:48.371848 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rvcrj" podStartSLOduration=1.9771282110000001 podStartE2EDuration="5.371822811s" totalImagesPullingTime="3.3946946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 02:54:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:54:48.343259166 +0000 UTC m=+347.190088479" watchObservedRunningTime="2026-09-10 02:54:48.371822811 +0000 UTC m=+347.218652079" Sep 10 02:54:48.372451 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:48.372400 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c" podStartSLOduration=1.683965559 podStartE2EDuration="5.372384076s" totalImagesPullingTime="3.688418517s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 02:54:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:54:48.370173674 +0000 UTC m=+347.217002934" watchObservedRunningTime="2026-09-10 02:54:48.372384076 +0000 UTC m=+347.219213345" Sep 10 02:54:48.715598 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:48.715555 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c" Sep 10 02:54:48.720344 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:48.720316 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c" Sep 10 02:54:49.320178 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:49.320145 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c" Sep 10 02:54:49.320578 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:49.320559 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rvcrj" Sep 10 02:54:49.321180 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:54:49.321164 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c" Sep 10 02:57:06.067285 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:06.067246 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-tx5pt"] Sep 10 02:57:06.071277 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:06.071254 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tx5pt" Sep 10 02:57:06.073617 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:06.073596 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 02:57:06.073720 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:06.073654 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Sep 10 02:57:06.074300 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:06.074279 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Sep 10 02:57:06.074300 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:06.074292 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-j6l5h\"" Sep 10 02:57:06.080235 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:06.080214 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-tx5pt"] Sep 10 02:57:06.082220 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:06.082196 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e57b24ec-2ac3-422a-a51d-59c6eb668703-data\") pod \"postgres-69fb4594cf-tx5pt\" (UID: \"e57b24ec-2ac3-422a-a51d-59c6eb668703\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tx5pt" Sep 10 02:57:06.082305 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:06.082269 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lz7xk\" (UniqueName: \"kubernetes.io/projected/e57b24ec-2ac3-422a-a51d-59c6eb668703-kube-api-access-lz7xk\") pod \"postgres-69fb4594cf-tx5pt\" (UID: \"e57b24ec-2ac3-422a-a51d-59c6eb668703\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tx5pt" Sep 10 02:57:06.183661 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:06.183629 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e57b24ec-2ac3-422a-a51d-59c6eb668703-data\") pod \"postgres-69fb4594cf-tx5pt\" (UID: \"e57b24ec-2ac3-422a-a51d-59c6eb668703\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tx5pt" Sep 10 02:57:06.183849 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:06.183691 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz7xk\" (UniqueName: \"kubernetes.io/projected/e57b24ec-2ac3-422a-a51d-59c6eb668703-kube-api-access-lz7xk\") pod \"postgres-69fb4594cf-tx5pt\" (UID: \"e57b24ec-2ac3-422a-a51d-59c6eb668703\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tx5pt" Sep 10 02:57:06.184044 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:06.184018 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e57b24ec-2ac3-422a-a51d-59c6eb668703-data\") pod \"postgres-69fb4594cf-tx5pt\" (UID: \"e57b24ec-2ac3-422a-a51d-59c6eb668703\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tx5pt" Sep 10 02:57:06.192228 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:06.192208 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lz7xk\" (UniqueName: \"kubernetes.io/projected/e57b24ec-2ac3-422a-a51d-59c6eb668703-kube-api-access-lz7xk\") pod \"postgres-69fb4594cf-tx5pt\" (UID: \"e57b24ec-2ac3-422a-a51d-59c6eb668703\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tx5pt" Sep 10 02:57:06.383904 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:06.383872 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tx5pt" Sep 10 02:57:06.653452 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:06.653371 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-tx5pt"] Sep 10 02:57:06.656105 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:57:06.656073 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode57b24ec_2ac3_422a_a51d_59c6eb668703.slice/crio-2d2e5d7511ad9af37c90dd3e2e55b2872ed303b7ddcf9228617e4c00b7ac5bdc WatchSource:0}: Error finding container 2d2e5d7511ad9af37c90dd3e2e55b2872ed303b7ddcf9228617e4c00b7ac5bdc: Status 404 returned error can't find the container with id 2d2e5d7511ad9af37c90dd3e2e55b2872ed303b7ddcf9228617e4c00b7ac5bdc Sep 10 02:57:06.947470 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:06.947388 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tx5pt" event={"ID":"e57b24ec-2ac3-422a-a51d-59c6eb668703","Type":"ContainerStarted","Data":"2d2e5d7511ad9af37c90dd3e2e55b2872ed303b7ddcf9228617e4c00b7ac5bdc"} Sep 10 02:57:11.977765 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:11.977722 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tx5pt" event={"ID":"e57b24ec-2ac3-422a-a51d-59c6eb668703","Type":"ContainerStarted","Data":"5000b3786b612ded9c2e3b1886e59aede8a527864198476b1dffb48027685cbb"} Sep 10 02:57:11.978272 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:11.977882 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tx5pt" Sep 10 02:57:11.994990 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:11.994942 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tx5pt" podStartSLOduration=0.860850933 podStartE2EDuration="5.994926742s" totalImagesPullingTime="5.134075809s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 02:57:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:57:11.994048023 +0000 UTC m=+490.840877291" watchObservedRunningTime="2026-09-10 02:57:11.994926742 +0000 UTC m=+490.841756009" Sep 10 02:57:18.011805 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:18.011774 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tx5pt" Sep 10 02:57:25.323382 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:25.323347 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-675497b4fb-fd6tt"] Sep 10 02:57:25.326637 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:25.326621 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-675497b4fb-fd6tt" Sep 10 02:57:25.329608 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:25.329588 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 10 02:57:25.329801 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:25.329788 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 10 02:57:25.329882 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:25.329828 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 10 02:57:25.330351 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:25.330333 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-tnj45\"" Sep 10 02:57:25.336537 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:25.336494 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-675497b4fb-fd6tt"] Sep 10 02:57:25.462378 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:25.462340 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e25fd299-d6a1-4cc0-8213-50b1eca83118-metrics-certs\") pod \"maas-controller-675497b4fb-fd6tt\" (UID: \"e25fd299-d6a1-4cc0-8213-50b1eca83118\") " pod="opendatahub/maas-controller-675497b4fb-fd6tt" Sep 10 02:57:25.462595 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:25.462462 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e25fd299-d6a1-4cc0-8213-50b1eca83118-webhook-cert\") pod \"maas-controller-675497b4fb-fd6tt\" (UID: \"e25fd299-d6a1-4cc0-8213-50b1eca83118\") " pod="opendatahub/maas-controller-675497b4fb-fd6tt" Sep 10 02:57:25.462595 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:25.462539 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dvh6\" (UniqueName: \"kubernetes.io/projected/e25fd299-d6a1-4cc0-8213-50b1eca83118-kube-api-access-5dvh6\") pod \"maas-controller-675497b4fb-fd6tt\" (UID: \"e25fd299-d6a1-4cc0-8213-50b1eca83118\") " pod="opendatahub/maas-controller-675497b4fb-fd6tt" Sep 10 02:57:25.506856 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:25.506822 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-64885d587b-qjgxn"] Sep 10 02:57:25.510306 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:25.510291 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64885d587b-qjgxn" Sep 10 02:57:25.521964 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:25.521938 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64885d587b-qjgxn"] Sep 10 02:57:25.563322 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:25.563293 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/286a67b5-e0ad-4452-afdd-9e3443686520-webhook-cert\") pod \"maas-controller-64885d587b-qjgxn\" (UID: \"286a67b5-e0ad-4452-afdd-9e3443686520\") " pod="opendatahub/maas-controller-64885d587b-qjgxn" Sep 10 02:57:25.563484 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:25.563347 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/286a67b5-e0ad-4452-afdd-9e3443686520-metrics-certs\") pod \"maas-controller-64885d587b-qjgxn\" (UID: \"286a67b5-e0ad-4452-afdd-9e3443686520\") " pod="opendatahub/maas-controller-64885d587b-qjgxn" Sep 10 02:57:25.563484 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:25.563371 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e25fd299-d6a1-4cc0-8213-50b1eca83118-webhook-cert\") pod \"maas-controller-675497b4fb-fd6tt\" (UID: \"e25fd299-d6a1-4cc0-8213-50b1eca83118\") " pod="opendatahub/maas-controller-675497b4fb-fd6tt" Sep 10 02:57:25.563484 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:25.563447 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dvh6\" (UniqueName: \"kubernetes.io/projected/e25fd299-d6a1-4cc0-8213-50b1eca83118-kube-api-access-5dvh6\") pod \"maas-controller-675497b4fb-fd6tt\" (UID: \"e25fd299-d6a1-4cc0-8213-50b1eca83118\") " pod="opendatahub/maas-controller-675497b4fb-fd6tt" Sep 10 02:57:25.563678 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:25.563497 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e25fd299-d6a1-4cc0-8213-50b1eca83118-metrics-certs\") pod \"maas-controller-675497b4fb-fd6tt\" (UID: \"e25fd299-d6a1-4cc0-8213-50b1eca83118\") " pod="opendatahub/maas-controller-675497b4fb-fd6tt" Sep 10 02:57:25.563678 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:57:25.563601 2560 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found Sep 10 02:57:25.563678 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:25.563643 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nn5xg\" (UniqueName: \"kubernetes.io/projected/286a67b5-e0ad-4452-afdd-9e3443686520-kube-api-access-nn5xg\") pod \"maas-controller-64885d587b-qjgxn\" (UID: \"286a67b5-e0ad-4452-afdd-9e3443686520\") " pod="opendatahub/maas-controller-64885d587b-qjgxn" Sep 10 02:57:25.563678 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:57:25.563670 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e25fd299-d6a1-4cc0-8213-50b1eca83118-metrics-certs podName:e25fd299-d6a1-4cc0-8213-50b1eca83118 nodeName:}" failed. No retries permitted until 2026-09-10 02:57:26.0636488 +0000 UTC m=+504.910478050 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e25fd299-d6a1-4cc0-8213-50b1eca83118-metrics-certs") pod "maas-controller-675497b4fb-fd6tt" (UID: "e25fd299-d6a1-4cc0-8213-50b1eca83118") : secret "maas-controller-metrics-tls" not found Sep 10 02:57:25.565963 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:25.565935 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e25fd299-d6a1-4cc0-8213-50b1eca83118-webhook-cert\") pod \"maas-controller-675497b4fb-fd6tt\" (UID: \"e25fd299-d6a1-4cc0-8213-50b1eca83118\") " pod="opendatahub/maas-controller-675497b4fb-fd6tt" Sep 10 02:57:25.572810 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:25.572786 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dvh6\" (UniqueName: \"kubernetes.io/projected/e25fd299-d6a1-4cc0-8213-50b1eca83118-kube-api-access-5dvh6\") pod \"maas-controller-675497b4fb-fd6tt\" (UID: \"e25fd299-d6a1-4cc0-8213-50b1eca83118\") " pod="opendatahub/maas-controller-675497b4fb-fd6tt" Sep 10 02:57:25.664947 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:25.664910 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn5xg\" (UniqueName: \"kubernetes.io/projected/286a67b5-e0ad-4452-afdd-9e3443686520-kube-api-access-nn5xg\") pod \"maas-controller-64885d587b-qjgxn\" (UID: \"286a67b5-e0ad-4452-afdd-9e3443686520\") " pod="opendatahub/maas-controller-64885d587b-qjgxn" Sep 10 02:57:25.665108 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:25.664965 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/286a67b5-e0ad-4452-afdd-9e3443686520-webhook-cert\") pod \"maas-controller-64885d587b-qjgxn\" (UID: \"286a67b5-e0ad-4452-afdd-9e3443686520\") " pod="opendatahub/maas-controller-64885d587b-qjgxn" Sep 10 02:57:25.665108 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:25.665016 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/286a67b5-e0ad-4452-afdd-9e3443686520-metrics-certs\") pod \"maas-controller-64885d587b-qjgxn\" (UID: \"286a67b5-e0ad-4452-afdd-9e3443686520\") " pod="opendatahub/maas-controller-64885d587b-qjgxn" Sep 10 02:57:25.667659 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:25.667635 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/286a67b5-e0ad-4452-afdd-9e3443686520-metrics-certs\") pod \"maas-controller-64885d587b-qjgxn\" (UID: \"286a67b5-e0ad-4452-afdd-9e3443686520\") " pod="opendatahub/maas-controller-64885d587b-qjgxn" Sep 10 02:57:25.667770 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:25.667682 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/286a67b5-e0ad-4452-afdd-9e3443686520-webhook-cert\") pod \"maas-controller-64885d587b-qjgxn\" (UID: \"286a67b5-e0ad-4452-afdd-9e3443686520\") " pod="opendatahub/maas-controller-64885d587b-qjgxn" Sep 10 02:57:25.674364 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:25.674338 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nn5xg\" (UniqueName: \"kubernetes.io/projected/286a67b5-e0ad-4452-afdd-9e3443686520-kube-api-access-nn5xg\") pod \"maas-controller-64885d587b-qjgxn\" (UID: \"286a67b5-e0ad-4452-afdd-9e3443686520\") " pod="opendatahub/maas-controller-64885d587b-qjgxn" Sep 10 02:57:25.820873 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:25.820843 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64885d587b-qjgxn" Sep 10 02:57:26.068673 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:26.068579 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e25fd299-d6a1-4cc0-8213-50b1eca83118-metrics-certs\") pod \"maas-controller-675497b4fb-fd6tt\" (UID: \"e25fd299-d6a1-4cc0-8213-50b1eca83118\") " pod="opendatahub/maas-controller-675497b4fb-fd6tt" Sep 10 02:57:26.071278 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:26.071252 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e25fd299-d6a1-4cc0-8213-50b1eca83118-metrics-certs\") pod \"maas-controller-675497b4fb-fd6tt\" (UID: \"e25fd299-d6a1-4cc0-8213-50b1eca83118\") " pod="opendatahub/maas-controller-675497b4fb-fd6tt" Sep 10 02:57:26.092825 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:26.092802 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64885d587b-qjgxn"] Sep 10 02:57:26.094782 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:57:26.094755 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod286a67b5_e0ad_4452_afdd_9e3443686520.slice/crio-609cba5964a73f99d86d27e5a326e99ff9287a61eddf6632254bdf242d5de8da WatchSource:0}: Error finding container 609cba5964a73f99d86d27e5a326e99ff9287a61eddf6632254bdf242d5de8da: Status 404 returned error can't find the container with id 609cba5964a73f99d86d27e5a326e99ff9287a61eddf6632254bdf242d5de8da Sep 10 02:57:26.237296 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:26.237257 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-675497b4fb-fd6tt" Sep 10 02:57:26.507563 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:26.507536 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-675497b4fb-fd6tt"] Sep 10 02:57:26.510287 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:57:26.510258 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode25fd299_d6a1_4cc0_8213_50b1eca83118.slice/crio-a1788b3b53abf0e9bb3c4e1c24b6f4cf0ea7247d1f84f2eb1dc2d4defe09e86b WatchSource:0}: Error finding container a1788b3b53abf0e9bb3c4e1c24b6f4cf0ea7247d1f84f2eb1dc2d4defe09e86b: Status 404 returned error can't find the container with id a1788b3b53abf0e9bb3c4e1c24b6f4cf0ea7247d1f84f2eb1dc2d4defe09e86b Sep 10 02:57:27.055793 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:27.055755 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-675497b4fb-fd6tt" event={"ID":"e25fd299-d6a1-4cc0-8213-50b1eca83118","Type":"ContainerStarted","Data":"a1788b3b53abf0e9bb3c4e1c24b6f4cf0ea7247d1f84f2eb1dc2d4defe09e86b"} Sep 10 02:57:27.058001 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:27.057973 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64885d587b-qjgxn" event={"ID":"286a67b5-e0ad-4452-afdd-9e3443686520","Type":"ContainerStarted","Data":"609cba5964a73f99d86d27e5a326e99ff9287a61eddf6632254bdf242d5de8da"} Sep 10 02:57:29.069262 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:29.069154 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-675497b4fb-fd6tt" event={"ID":"e25fd299-d6a1-4cc0-8213-50b1eca83118","Type":"ContainerStarted","Data":"a6e62167814780f382107ea5f5e2e7d59db374515adcdd54d52575ab354a799d"} Sep 10 02:57:29.069682 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:29.069352 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-675497b4fb-fd6tt" Sep 10 02:57:29.070607 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:29.070581 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64885d587b-qjgxn" event={"ID":"286a67b5-e0ad-4452-afdd-9e3443686520","Type":"ContainerStarted","Data":"b2b6ba8e718834750a8caa70111337ac3ceec2d8e2580facd248c7ea283bf1f3"} Sep 10 02:57:29.070732 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:29.070719 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-64885d587b-qjgxn" Sep 10 02:57:29.087013 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:29.086966 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-675497b4fb-fd6tt" podStartSLOduration=1.965264804 podStartE2EDuration="4.086953229s" totalImagesPullingTime="2.121688425s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 02:57:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:57:29.084894126 +0000 UTC m=+507.931723386" watchObservedRunningTime="2026-09-10 02:57:29.086953229 +0000 UTC m=+507.933782497" Sep 10 02:57:29.102268 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:29.102221 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-64885d587b-qjgxn" podStartSLOduration=1.571256006 podStartE2EDuration="4.102207094s" totalImagesPullingTime="2.530951088s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 02:57:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:57:29.101677349 +0000 UTC m=+507.948506618" watchObservedRunningTime="2026-09-10 02:57:29.102207094 +0000 UTC m=+507.949036363" Sep 10 02:57:40.077659 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:40.077631 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-64885d587b-qjgxn" Sep 10 02:57:40.078033 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:40.077708 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-675497b4fb-fd6tt" Sep 10 02:57:40.144418 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:40.144386 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-675497b4fb-fd6tt"] Sep 10 02:57:40.144640 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:40.144598 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-675497b4fb-fd6tt" podUID="e25fd299-d6a1-4cc0-8213-50b1eca83118" containerName="manager" containerID="cri-o://a6e62167814780f382107ea5f5e2e7d59db374515adcdd54d52575ab354a799d" gracePeriod=30 Sep 10 02:57:40.390174 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:40.390149 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-675497b4fb-fd6tt" Sep 10 02:57:40.502800 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:40.502761 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e25fd299-d6a1-4cc0-8213-50b1eca83118-webhook-cert\") pod \"e25fd299-d6a1-4cc0-8213-50b1eca83118\" (UID: \"e25fd299-d6a1-4cc0-8213-50b1eca83118\") " Sep 10 02:57:40.502800 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:40.502808 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e25fd299-d6a1-4cc0-8213-50b1eca83118-metrics-certs\") pod \"e25fd299-d6a1-4cc0-8213-50b1eca83118\" (UID: \"e25fd299-d6a1-4cc0-8213-50b1eca83118\") " Sep 10 02:57:40.503004 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:40.502854 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5dvh6\" (UniqueName: \"kubernetes.io/projected/e25fd299-d6a1-4cc0-8213-50b1eca83118-kube-api-access-5dvh6\") pod \"e25fd299-d6a1-4cc0-8213-50b1eca83118\" (UID: \"e25fd299-d6a1-4cc0-8213-50b1eca83118\") " Sep 10 02:57:40.505242 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:40.505205 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e25fd299-d6a1-4cc0-8213-50b1eca83118-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "e25fd299-d6a1-4cc0-8213-50b1eca83118" (UID: "e25fd299-d6a1-4cc0-8213-50b1eca83118"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 02:57:40.505364 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:40.505254 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e25fd299-d6a1-4cc0-8213-50b1eca83118-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e25fd299-d6a1-4cc0-8213-50b1eca83118" (UID: "e25fd299-d6a1-4cc0-8213-50b1eca83118"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 02:57:40.505364 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:40.505250 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e25fd299-d6a1-4cc0-8213-50b1eca83118-kube-api-access-5dvh6" (OuterVolumeSpecName: "kube-api-access-5dvh6") pod "e25fd299-d6a1-4cc0-8213-50b1eca83118" (UID: "e25fd299-d6a1-4cc0-8213-50b1eca83118"). InnerVolumeSpecName "kube-api-access-5dvh6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 02:57:40.604023 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:40.603983 2560 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e25fd299-d6a1-4cc0-8213-50b1eca83118-webhook-cert\") on node \"ip-10-0-133-37.ec2.internal\" DevicePath \"\"" Sep 10 02:57:40.604023 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:40.604016 2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e25fd299-d6a1-4cc0-8213-50b1eca83118-metrics-certs\") on node \"ip-10-0-133-37.ec2.internal\" DevicePath \"\"" Sep 10 02:57:40.604023 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:40.604025 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5dvh6\" (UniqueName: \"kubernetes.io/projected/e25fd299-d6a1-4cc0-8213-50b1eca83118-kube-api-access-5dvh6\") on node \"ip-10-0-133-37.ec2.internal\" DevicePath \"\"" Sep 10 02:57:41.131973 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:41.131934 2560 generic.go:356] "Generic (PLEG): container finished" podID="e25fd299-d6a1-4cc0-8213-50b1eca83118" containerID="a6e62167814780f382107ea5f5e2e7d59db374515adcdd54d52575ab354a799d" exitCode=0 Sep 10 02:57:41.132380 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:41.131980 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-675497b4fb-fd6tt" Sep 10 02:57:41.132380 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:41.132008 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-675497b4fb-fd6tt" event={"ID":"e25fd299-d6a1-4cc0-8213-50b1eca83118","Type":"ContainerDied","Data":"a6e62167814780f382107ea5f5e2e7d59db374515adcdd54d52575ab354a799d"} Sep 10 02:57:41.132380 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:41.132060 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-675497b4fb-fd6tt" event={"ID":"e25fd299-d6a1-4cc0-8213-50b1eca83118","Type":"ContainerDied","Data":"a1788b3b53abf0e9bb3c4e1c24b6f4cf0ea7247d1f84f2eb1dc2d4defe09e86b"} Sep 10 02:57:41.132380 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:41.132082 2560 scope.go:117] "RemoveContainer" containerID="a6e62167814780f382107ea5f5e2e7d59db374515adcdd54d52575ab354a799d" Sep 10 02:57:41.141598 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:41.141581 2560 scope.go:117] "RemoveContainer" containerID="a6e62167814780f382107ea5f5e2e7d59db374515adcdd54d52575ab354a799d" Sep 10 02:57:41.141918 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:57:41.141901 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6e62167814780f382107ea5f5e2e7d59db374515adcdd54d52575ab354a799d\": container with ID starting with a6e62167814780f382107ea5f5e2e7d59db374515adcdd54d52575ab354a799d not found: ID does not exist" containerID="a6e62167814780f382107ea5f5e2e7d59db374515adcdd54d52575ab354a799d" Sep 10 02:57:41.141966 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:41.141927 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6e62167814780f382107ea5f5e2e7d59db374515adcdd54d52575ab354a799d"} err="failed to get container status \"a6e62167814780f382107ea5f5e2e7d59db374515adcdd54d52575ab354a799d\": rpc error: code = NotFound desc = could not find container \"a6e62167814780f382107ea5f5e2e7d59db374515adcdd54d52575ab354a799d\": container with ID starting with a6e62167814780f382107ea5f5e2e7d59db374515adcdd54d52575ab354a799d not found: ID does not exist" Sep 10 02:57:41.152863 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:41.152840 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-675497b4fb-fd6tt"] Sep 10 02:57:41.158389 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:41.158367 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-675497b4fb-fd6tt"] Sep 10 02:57:41.645832 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:57:41.645795 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e25fd299-d6a1-4cc0-8213-50b1eca83118" path="/var/lib/kubelet/pods/e25fd299-d6a1-4cc0-8213-50b1eca83118/volumes" Sep 10 02:58:02.547158 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:02.547125 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-vzvn2"] Sep 10 02:58:02.547571 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:02.547487 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e25fd299-d6a1-4cc0-8213-50b1eca83118" containerName="manager" Sep 10 02:58:02.547571 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:02.547498 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e25fd299-d6a1-4cc0-8213-50b1eca83118" containerName="manager" Sep 10 02:58:02.547649 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:02.547630 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e25fd299-d6a1-4cc0-8213-50b1eca83118" containerName="manager" Sep 10 02:58:02.553421 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:02.553396 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-vzvn2" Sep 10 02:58:02.556782 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:02.556760 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Sep 10 02:58:02.556907 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:02.556885 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-lmgtp\"" Sep 10 02:58:02.562137 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:02.562116 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-vzvn2"] Sep 10 02:58:02.571790 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:02.571765 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15783e7b-43bb-4ee1-a4fe-bb71e22d7a69-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-vzvn2\" (UID: \"15783e7b-43bb-4ee1-a4fe-bb71e22d7a69\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-vzvn2" Sep 10 02:58:02.571911 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:02.571825 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8v4b\" (UniqueName: \"kubernetes.io/projected/15783e7b-43bb-4ee1-a4fe-bb71e22d7a69-kube-api-access-m8v4b\") pod \"payload-pre-processing-59fbc47bf8-vzvn2\" (UID: \"15783e7b-43bb-4ee1-a4fe-bb71e22d7a69\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-vzvn2" Sep 10 02:58:02.585050 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:02.585025 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-fh5st"] Sep 10 02:58:02.588224 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:02.588211 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-fh5st" Sep 10 02:58:02.599296 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:02.599274 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-fh5st"] Sep 10 02:58:02.672691 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:02.672651 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d268b46e-913a-42c9-a7a5-816c01c61848-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-fh5st\" (UID: \"d268b46e-913a-42c9-a7a5-816c01c61848\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-fh5st" Sep 10 02:58:02.672878 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:02.672748 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8v4b\" (UniqueName: \"kubernetes.io/projected/15783e7b-43bb-4ee1-a4fe-bb71e22d7a69-kube-api-access-m8v4b\") pod \"payload-pre-processing-59fbc47bf8-vzvn2\" (UID: \"15783e7b-43bb-4ee1-a4fe-bb71e22d7a69\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-vzvn2" Sep 10 02:58:02.672878 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:02.672853 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15783e7b-43bb-4ee1-a4fe-bb71e22d7a69-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-vzvn2\" (UID: \"15783e7b-43bb-4ee1-a4fe-bb71e22d7a69\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-vzvn2" Sep 10 02:58:02.672878 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:02.672878 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbrpb\" (UniqueName: \"kubernetes.io/projected/d268b46e-913a-42c9-a7a5-816c01c61848-kube-api-access-fbrpb\") pod \"payload-processing-5b54b7fbb7-fh5st\" (UID: \"d268b46e-913a-42c9-a7a5-816c01c61848\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-fh5st" Sep 10 02:58:02.673660 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:02.673639 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15783e7b-43bb-4ee1-a4fe-bb71e22d7a69-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-vzvn2\" (UID: \"15783e7b-43bb-4ee1-a4fe-bb71e22d7a69\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-vzvn2" Sep 10 02:58:02.683372 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:02.683349 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8v4b\" (UniqueName: \"kubernetes.io/projected/15783e7b-43bb-4ee1-a4fe-bb71e22d7a69-kube-api-access-m8v4b\") pod \"payload-pre-processing-59fbc47bf8-vzvn2\" (UID: \"15783e7b-43bb-4ee1-a4fe-bb71e22d7a69\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-vzvn2" Sep 10 02:58:02.773392 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:02.773356 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbrpb\" (UniqueName: \"kubernetes.io/projected/d268b46e-913a-42c9-a7a5-816c01c61848-kube-api-access-fbrpb\") pod \"payload-processing-5b54b7fbb7-fh5st\" (UID: \"d268b46e-913a-42c9-a7a5-816c01c61848\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-fh5st" Sep 10 02:58:02.773590 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:02.773406 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d268b46e-913a-42c9-a7a5-816c01c61848-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-fh5st\" (UID: \"d268b46e-913a-42c9-a7a5-816c01c61848\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-fh5st" Sep 10 02:58:02.774083 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:02.774064 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d268b46e-913a-42c9-a7a5-816c01c61848-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-fh5st\" (UID: \"d268b46e-913a-42c9-a7a5-816c01c61848\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-fh5st" Sep 10 02:58:02.783611 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:02.783584 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbrpb\" (UniqueName: \"kubernetes.io/projected/d268b46e-913a-42c9-a7a5-816c01c61848-kube-api-access-fbrpb\") pod \"payload-processing-5b54b7fbb7-fh5st\" (UID: \"d268b46e-913a-42c9-a7a5-816c01c61848\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-fh5st" Sep 10 02:58:02.863733 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:02.863702 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-vzvn2" Sep 10 02:58:02.899044 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:02.899006 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-fh5st" Sep 10 02:58:03.138305 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:03.138279 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-vzvn2"] Sep 10 02:58:03.140304 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:58:03.140279 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15783e7b_43bb_4ee1_a4fe_bb71e22d7a69.slice/crio-c0482c91b30291af380fd8a43ac3374464d1d4636eef340550a7876a929078ce WatchSource:0}: Error finding container c0482c91b30291af380fd8a43ac3374464d1d4636eef340550a7876a929078ce: Status 404 returned error can't find the container with id c0482c91b30291af380fd8a43ac3374464d1d4636eef340550a7876a929078ce Sep 10 02:58:03.183779 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:03.183755 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-fh5st"] Sep 10 02:58:03.186195 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:58:03.186167 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd268b46e_913a_42c9_a7a5_816c01c61848.slice/crio-274619edc5f41b53e2cf10c45334a0d5b7a1e461e0321aca2c7be18c0567bfb3 WatchSource:0}: Error finding container 274619edc5f41b53e2cf10c45334a0d5b7a1e461e0321aca2c7be18c0567bfb3: Status 404 returned error can't find the container with id 274619edc5f41b53e2cf10c45334a0d5b7a1e461e0321aca2c7be18c0567bfb3 Sep 10 02:58:03.237562 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:03.237498 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-fh5st" event={"ID":"d268b46e-913a-42c9-a7a5-816c01c61848","Type":"ContainerStarted","Data":"274619edc5f41b53e2cf10c45334a0d5b7a1e461e0321aca2c7be18c0567bfb3"} Sep 10 02:58:03.238744 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:03.238713 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-vzvn2" event={"ID":"15783e7b-43bb-4ee1-a4fe-bb71e22d7a69","Type":"ContainerStarted","Data":"c0482c91b30291af380fd8a43ac3374464d1d4636eef340550a7876a929078ce"} Sep 10 02:58:06.264813 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:06.264775 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-fh5st" event={"ID":"d268b46e-913a-42c9-a7a5-816c01c61848","Type":"ContainerStarted","Data":"16ac1ab733adde16c8332b8b08fe7e0380a8f886323e93a6e03fa8a8586729b0"} Sep 10 02:58:06.265304 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:06.264891 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-fh5st" Sep 10 02:58:06.266383 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:06.266356 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-vzvn2" event={"ID":"15783e7b-43bb-4ee1-a4fe-bb71e22d7a69","Type":"ContainerStarted","Data":"761673c689dc9278562e95dd74952ec75478c6d5104177574ca971393842c153"} Sep 10 02:58:06.266490 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:06.266480 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-vzvn2" Sep 10 02:58:06.283253 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:06.283203 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-fh5st" podStartSLOduration=1.811307958 podStartE2EDuration="4.283184658s" totalImagesPullingTime="2.4718767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 02:58:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:58:06.280898127 +0000 UTC m=+545.127727421" watchObservedRunningTime="2026-09-10 02:58:06.283184658 +0000 UTC m=+545.130013928" Sep 10 02:58:06.298872 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:06.298823 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-vzvn2" podStartSLOduration=1.7749587500000001 podStartE2EDuration="4.29880213s" totalImagesPullingTime="2.52384338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 02:58:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:58:06.298481779 +0000 UTC m=+545.145311048" watchObservedRunningTime="2026-09-10 02:58:06.29880213 +0000 UTC m=+545.145631398" Sep 10 02:58:17.274033 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:17.274000 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-fh5st" Sep 10 02:58:17.274621 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:17.274160 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-vzvn2" Sep 10 02:58:45.970945 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:45.970895 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x"] Sep 10 02:58:45.979031 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:45.979007 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x" Sep 10 02:58:45.981662 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:45.981635 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-c9khk\"" Sep 10 02:58:45.982638 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:45.982618 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\"" Sep 10 02:58:45.982749 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:45.982618 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 10 02:58:45.982749 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:45.982643 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 10 02:58:45.994086 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:45.990074 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x"] Sep 10 02:58:46.074281 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:46.074246 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87m9l\" (UniqueName: \"kubernetes.io/projected/fba4f586-87c6-41ab-a94e-a50ffce60176-kube-api-access-87m9l\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x\" (UID: \"fba4f586-87c6-41ab-a94e-a50ffce60176\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x" Sep 10 02:58:46.074281 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:46.074288 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fba4f586-87c6-41ab-a94e-a50ffce60176-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x\" (UID: \"fba4f586-87c6-41ab-a94e-a50ffce60176\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x" Sep 10 02:58:46.074493 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:46.074367 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fba4f586-87c6-41ab-a94e-a50ffce60176-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x\" (UID: \"fba4f586-87c6-41ab-a94e-a50ffce60176\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x" Sep 10 02:58:46.074493 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:46.074417 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fba4f586-87c6-41ab-a94e-a50ffce60176-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x\" (UID: \"fba4f586-87c6-41ab-a94e-a50ffce60176\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x" Sep 10 02:58:46.074493 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:46.074432 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fba4f586-87c6-41ab-a94e-a50ffce60176-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x\" (UID: \"fba4f586-87c6-41ab-a94e-a50ffce60176\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x" Sep 10 02:58:46.074493 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:46.074447 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fba4f586-87c6-41ab-a94e-a50ffce60176-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x\" (UID: \"fba4f586-87c6-41ab-a94e-a50ffce60176\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x" Sep 10 02:58:46.175929 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:46.175887 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fba4f586-87c6-41ab-a94e-a50ffce60176-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x\" (UID: \"fba4f586-87c6-41ab-a94e-a50ffce60176\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x" Sep 10 02:58:46.176120 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:46.175974 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fba4f586-87c6-41ab-a94e-a50ffce60176-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x\" (UID: \"fba4f586-87c6-41ab-a94e-a50ffce60176\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x" Sep 10 02:58:46.176120 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:46.175999 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fba4f586-87c6-41ab-a94e-a50ffce60176-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x\" (UID: \"fba4f586-87c6-41ab-a94e-a50ffce60176\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x" Sep 10 02:58:46.176120 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:46.176025 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fba4f586-87c6-41ab-a94e-a50ffce60176-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x\" (UID: \"fba4f586-87c6-41ab-a94e-a50ffce60176\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x" Sep 10 02:58:46.176120 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:46.176070 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87m9l\" (UniqueName: \"kubernetes.io/projected/fba4f586-87c6-41ab-a94e-a50ffce60176-kube-api-access-87m9l\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x\" (UID: \"fba4f586-87c6-41ab-a94e-a50ffce60176\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x" Sep 10 02:58:46.176120 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:46.176095 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fba4f586-87c6-41ab-a94e-a50ffce60176-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x\" (UID: \"fba4f586-87c6-41ab-a94e-a50ffce60176\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x" Sep 10 02:58:46.176380 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:46.176264 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fba4f586-87c6-41ab-a94e-a50ffce60176-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x\" (UID: \"fba4f586-87c6-41ab-a94e-a50ffce60176\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x" Sep 10 02:58:46.176436 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:46.176375 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fba4f586-87c6-41ab-a94e-a50ffce60176-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x\" (UID: \"fba4f586-87c6-41ab-a94e-a50ffce60176\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x" Sep 10 02:58:46.176491 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:46.176477 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fba4f586-87c6-41ab-a94e-a50ffce60176-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x\" (UID: \"fba4f586-87c6-41ab-a94e-a50ffce60176\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x" Sep 10 02:58:46.178342 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:46.178317 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fba4f586-87c6-41ab-a94e-a50ffce60176-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x\" (UID: \"fba4f586-87c6-41ab-a94e-a50ffce60176\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x" Sep 10 02:58:46.178783 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:46.178764 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fba4f586-87c6-41ab-a94e-a50ffce60176-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x\" (UID: \"fba4f586-87c6-41ab-a94e-a50ffce60176\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x" Sep 10 02:58:46.183852 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:46.183830 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87m9l\" (UniqueName: \"kubernetes.io/projected/fba4f586-87c6-41ab-a94e-a50ffce60176-kube-api-access-87m9l\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x\" (UID: \"fba4f586-87c6-41ab-a94e-a50ffce60176\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x" Sep 10 02:58:46.295139 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:46.295052 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x" Sep 10 02:58:46.565534 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:46.565489 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x"] Sep 10 02:58:46.568271 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:58:46.568246 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfba4f586_87c6_41ab_a94e_a50ffce60176.slice/crio-ccfe1ab0800311d804dd0a389ee71ca82ba8ab0c8c3213cab6afe819d3684e9e WatchSource:0}: Error finding container ccfe1ab0800311d804dd0a389ee71ca82ba8ab0c8c3213cab6afe819d3684e9e: Status 404 returned error can't find the container with id ccfe1ab0800311d804dd0a389ee71ca82ba8ab0c8c3213cab6afe819d3684e9e Sep 10 02:58:47.464105 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:47.464071 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x" event={"ID":"fba4f586-87c6-41ab-a94e-a50ffce60176","Type":"ContainerStarted","Data":"ccfe1ab0800311d804dd0a389ee71ca82ba8ab0c8c3213cab6afe819d3684e9e"} Sep 10 02:58:48.470560 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:48.470523 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x" event={"ID":"fba4f586-87c6-41ab-a94e-a50ffce60176","Type":"ContainerStarted","Data":"2fe33a287a25dec7951c5646b58c8264bed259f5a11e40036f9e68460df5c82d"} Sep 10 02:58:48.488976 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:48.488924 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x" podStartSLOduration=2.459401299 podStartE2EDuration="3.488908764s" totalImagesPullingTime="1.029507465s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 02:58:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 02:58:48.487702284 +0000 UTC m=+587.334531553" watchObservedRunningTime="2026-09-10 02:58:48.488908764 +0000 UTC m=+587.335738031" Sep 10 02:58:56.295959 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:56.295924 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x" Sep 10 02:58:56.296379 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:56.295991 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x" Sep 10 02:58:56.308375 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:56.308348 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x" Sep 10 02:58:56.521186 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:58:56.521154 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x" Sep 10 02:59:01.565384 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:01.565354 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ghs46_a5210aa8-4171-4cc3-b65b-6237a9aee1d6/console-operator/2.log" Sep 10 02:59:01.565889 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:01.565864 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ghs46_a5210aa8-4171-4cc3-b65b-6237a9aee1d6/console-operator/2.log" Sep 10 02:59:01.571246 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:01.571227 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swhbm_8a6680c4-21b8-4fe2-a763-e84858c7ef90/ovn-acl-logging/0.log" Sep 10 02:59:01.571881 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:01.571864 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swhbm_8a6680c4-21b8-4fe2-a763-e84858c7ef90/ovn-acl-logging/0.log" Sep 10 02:59:21.668066 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:21.668029 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2"] Sep 10 02:59:21.671984 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:21.671964 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 02:59:21.674690 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:21.674669 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Sep 10 02:59:21.679877 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:21.679836 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2"] Sep 10 02:59:21.827130 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:21.827088 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6722\" (UniqueName: \"kubernetes.io/projected/e4195eb0-a460-48af-946a-7657e36ea1f6-kube-api-access-s6722\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2\" (UID: \"e4195eb0-a460-48af-946a-7657e36ea1f6\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 02:59:21.827315 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:21.827153 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4195eb0-a460-48af-946a-7657e36ea1f6-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2\" (UID: \"e4195eb0-a460-48af-946a-7657e36ea1f6\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 02:59:21.827315 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:21.827220 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4195eb0-a460-48af-946a-7657e36ea1f6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2\" (UID: \"e4195eb0-a460-48af-946a-7657e36ea1f6\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 02:59:21.827315 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:21.827242 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4195eb0-a460-48af-946a-7657e36ea1f6-home\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2\" (UID: \"e4195eb0-a460-48af-946a-7657e36ea1f6\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 02:59:21.827315 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:21.827303 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4195eb0-a460-48af-946a-7657e36ea1f6-dshm\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2\" (UID: \"e4195eb0-a460-48af-946a-7657e36ea1f6\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 02:59:21.827449 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:21.827353 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4195eb0-a460-48af-946a-7657e36ea1f6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2\" (UID: \"e4195eb0-a460-48af-946a-7657e36ea1f6\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 02:59:21.928711 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:21.928615 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4195eb0-a460-48af-946a-7657e36ea1f6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2\" (UID: \"e4195eb0-a460-48af-946a-7657e36ea1f6\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 02:59:21.928711 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:21.928655 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4195eb0-a460-48af-946a-7657e36ea1f6-home\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2\" (UID: \"e4195eb0-a460-48af-946a-7657e36ea1f6\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 02:59:21.928711 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:21.928687 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4195eb0-a460-48af-946a-7657e36ea1f6-dshm\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2\" (UID: \"e4195eb0-a460-48af-946a-7657e36ea1f6\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 02:59:21.928993 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:21.928732 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4195eb0-a460-48af-946a-7657e36ea1f6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2\" (UID: \"e4195eb0-a460-48af-946a-7657e36ea1f6\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 02:59:21.928993 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:21.928769 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6722\" (UniqueName: \"kubernetes.io/projected/e4195eb0-a460-48af-946a-7657e36ea1f6-kube-api-access-s6722\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2\" (UID: \"e4195eb0-a460-48af-946a-7657e36ea1f6\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 02:59:21.928993 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:21.928906 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4195eb0-a460-48af-946a-7657e36ea1f6-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2\" (UID: \"e4195eb0-a460-48af-946a-7657e36ea1f6\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 02:59:21.929226 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:21.929074 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4195eb0-a460-48af-946a-7657e36ea1f6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2\" (UID: \"e4195eb0-a460-48af-946a-7657e36ea1f6\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 02:59:21.929226 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:21.929125 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4195eb0-a460-48af-946a-7657e36ea1f6-home\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2\" (UID: \"e4195eb0-a460-48af-946a-7657e36ea1f6\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 02:59:21.929324 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:21.929235 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4195eb0-a460-48af-946a-7657e36ea1f6-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2\" (UID: \"e4195eb0-a460-48af-946a-7657e36ea1f6\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 02:59:21.931130 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:21.931103 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4195eb0-a460-48af-946a-7657e36ea1f6-dshm\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2\" (UID: \"e4195eb0-a460-48af-946a-7657e36ea1f6\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 02:59:21.931357 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:21.931341 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4195eb0-a460-48af-946a-7657e36ea1f6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2\" (UID: \"e4195eb0-a460-48af-946a-7657e36ea1f6\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 02:59:21.937343 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:21.937319 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6722\" (UniqueName: \"kubernetes.io/projected/e4195eb0-a460-48af-946a-7657e36ea1f6-kube-api-access-s6722\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2\" (UID: \"e4195eb0-a460-48af-946a-7657e36ea1f6\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 02:59:21.985085 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:21.985055 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 02:59:22.255743 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:22.255719 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2"] Sep 10 02:59:22.258446 ip-10-0-133-37 kubenswrapper[2560]: W0910 02:59:22.258416 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4195eb0_a460_48af_946a_7657e36ea1f6.slice/crio-6e8d58a0e8ca6ee634c968a154a9bb8f205db10b9244857724a4ee1021d04e95 WatchSource:0}: Error finding container 6e8d58a0e8ca6ee634c968a154a9bb8f205db10b9244857724a4ee1021d04e95: Status 404 returned error can't find the container with id 6e8d58a0e8ca6ee634c968a154a9bb8f205db10b9244857724a4ee1021d04e95 Sep 10 02:59:22.260131 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:22.260111 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 02:59:22.640481 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:22.640444 2560 generic.go:356] "Generic (PLEG): container finished" podID="e4195eb0-a460-48af-946a-7657e36ea1f6" containerID="fe819d5ef4750135daa55785d3a8fbbe921ac43f1c40a70da51a41bda9a286e8" exitCode=0 Sep 10 02:59:22.640663 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:22.640485 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" event={"ID":"e4195eb0-a460-48af-946a-7657e36ea1f6","Type":"ContainerDied","Data":"fe819d5ef4750135daa55785d3a8fbbe921ac43f1c40a70da51a41bda9a286e8"} Sep 10 02:59:22.640663 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:22.640550 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" event={"ID":"e4195eb0-a460-48af-946a-7657e36ea1f6","Type":"ContainerStarted","Data":"6e8d58a0e8ca6ee634c968a154a9bb8f205db10b9244857724a4ee1021d04e95"} Sep 10 02:59:22.640747 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:22.640729 2560 scope.go:117] "RemoveContainer" containerID="fe819d5ef4750135daa55785d3a8fbbe921ac43f1c40a70da51a41bda9a286e8" Sep 10 02:59:22.937941 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:59:22.937902 2560 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4195eb0_a460_48af_946a_7657e36ea1f6.slice/crio-conmon-bfa745f0b52fad8413e429d22754ed3f3b056f193261554f29b0267837fa8bc9.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4195eb0_a460_48af_946a_7657e36ea1f6.slice/crio-bfa745f0b52fad8413e429d22754ed3f3b056f193261554f29b0267837fa8bc9.scope\": RecentStats: unable to find data in memory cache]" Sep 10 02:59:23.647811 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:23.647782 2560 generic.go:356] "Generic (PLEG): container finished" podID="e4195eb0-a460-48af-946a-7657e36ea1f6" containerID="bfa745f0b52fad8413e429d22754ed3f3b056f193261554f29b0267837fa8bc9" exitCode=0 Sep 10 02:59:23.647979 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:23.647816 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" event={"ID":"e4195eb0-a460-48af-946a-7657e36ea1f6","Type":"ContainerDied","Data":"bfa745f0b52fad8413e429d22754ed3f3b056f193261554f29b0267837fa8bc9"} Sep 10 02:59:23.647979 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:23.647861 2560 scope.go:117] "RemoveContainer" containerID="fe819d5ef4750135daa55785d3a8fbbe921ac43f1c40a70da51a41bda9a286e8" Sep 10 02:59:23.648153 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:23.648135 2560 scope.go:117] "RemoveContainer" containerID="bfa745f0b52fad8413e429d22754ed3f3b056f193261554f29b0267837fa8bc9" Sep 10 02:59:23.648349 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:59:23.648320 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2_llm(e4195eb0-a460-48af-946a-7657e36ea1f6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" podUID="e4195eb0-a460-48af-946a-7657e36ea1f6" Sep 10 02:59:24.654357 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:24.654326 2560 scope.go:117] "RemoveContainer" containerID="bfa745f0b52fad8413e429d22754ed3f3b056f193261554f29b0267837fa8bc9" Sep 10 02:59:24.654747 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:59:24.654491 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2_llm(e4195eb0-a460-48af-946a-7657e36ea1f6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" podUID="e4195eb0-a460-48af-946a-7657e36ea1f6" Sep 10 02:59:31.985336 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:31.985303 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 02:59:31.985800 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:31.985380 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 02:59:31.985800 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:31.985411 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 02:59:31.985800 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:31.985432 2560 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 02:59:31.985901 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:31.985877 2560 scope.go:117] "RemoveContainer" containerID="bfa745f0b52fad8413e429d22754ed3f3b056f193261554f29b0267837fa8bc9" Sep 10 02:59:31.986089 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:59:31.986067 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2_llm(e4195eb0-a460-48af-946a-7657e36ea1f6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" podUID="e4195eb0-a460-48af-946a-7657e36ea1f6" Sep 10 02:59:32.692816 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:32.692792 2560 scope.go:117] "RemoveContainer" containerID="bfa745f0b52fad8413e429d22754ed3f3b056f193261554f29b0267837fa8bc9" Sep 10 02:59:32.692996 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:59:32.692976 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2_llm(e4195eb0-a460-48af-946a-7657e36ea1f6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" podUID="e4195eb0-a460-48af-946a-7657e36ea1f6" Sep 10 02:59:46.640365 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:46.640336 2560 scope.go:117] "RemoveContainer" containerID="bfa745f0b52fad8413e429d22754ed3f3b056f193261554f29b0267837fa8bc9" Sep 10 02:59:47.763802 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:47.763774 2560 generic.go:356] "Generic (PLEG): container finished" podID="e4195eb0-a460-48af-946a-7657e36ea1f6" containerID="d19371a202fc551b2b41b95e24737fce30291ae2b9dd40d6bd2b4308d4326736" exitCode=0 Sep 10 02:59:47.764223 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:47.763829 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" event={"ID":"e4195eb0-a460-48af-946a-7657e36ea1f6","Type":"ContainerDied","Data":"d19371a202fc551b2b41b95e24737fce30291ae2b9dd40d6bd2b4308d4326736"} Sep 10 02:59:47.764223 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:47.763874 2560 scope.go:117] "RemoveContainer" containerID="bfa745f0b52fad8413e429d22754ed3f3b056f193261554f29b0267837fa8bc9" Sep 10 02:59:47.764303 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:47.764230 2560 scope.go:117] "RemoveContainer" containerID="d19371a202fc551b2b41b95e24737fce30291ae2b9dd40d6bd2b4308d4326736" Sep 10 02:59:47.764440 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:59:47.764420 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2_llm(e4195eb0-a460-48af-946a-7657e36ea1f6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" podUID="e4195eb0-a460-48af-946a-7657e36ea1f6" Sep 10 02:59:51.985557 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:51.985489 2560 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 02:59:51.985969 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:51.985601 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 02:59:51.985969 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:51.985628 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 02:59:51.985969 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:51.985650 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 02:59:51.986091 ip-10-0-133-37 kubenswrapper[2560]: I0910 02:59:51.986025 2560 scope.go:117] "RemoveContainer" containerID="d19371a202fc551b2b41b95e24737fce30291ae2b9dd40d6bd2b4308d4326736" Sep 10 02:59:51.986260 ip-10-0-133-37 kubenswrapper[2560]: E0910 02:59:51.986240 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2_llm(e4195eb0-a460-48af-946a-7657e36ea1f6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" podUID="e4195eb0-a460-48af-946a-7657e36ea1f6" Sep 10 03:00:00.144043 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:00:00.144008 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29816820-q8xx9"] Sep 10 03:00:00.147443 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:00:00.147426 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816820-q8xx9" Sep 10 03:00:00.149741 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:00:00.149716 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h68h8\"" Sep 10 03:00:00.153395 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:00:00.153371 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29816820-q8xx9"] Sep 10 03:00:00.159225 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:00:00.159203 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816820-q8xx9" Sep 10 03:00:00.429629 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:00:00.429551 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29816820-q8xx9"] Sep 10 03:00:00.432678 ip-10-0-133-37 kubenswrapper[2560]: W0910 03:00:00.432641 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5006779d_f38f_4cce_be70_9a1af6d7d8a3.slice/crio-0b3f29116a2b82adbb5be7c9a76774cf14b4c1a2a3511081df7a8852fb023eec WatchSource:0}: Error finding container 0b3f29116a2b82adbb5be7c9a76774cf14b4c1a2a3511081df7a8852fb023eec: Status 404 returned error can't find the container with id 0b3f29116a2b82adbb5be7c9a76774cf14b4c1a2a3511081df7a8852fb023eec Sep 10 03:00:00.829440 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:00:00.829355 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816820-q8xx9" event={"ID":"5006779d-f38f-4cce-be70-9a1af6d7d8a3","Type":"ContainerStarted","Data":"0b3f29116a2b82adbb5be7c9a76774cf14b4c1a2a3511081df7a8852fb023eec"} Sep 10 03:00:03.640363 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:00:03.640327 2560 scope.go:117] "RemoveContainer" containerID="d19371a202fc551b2b41b95e24737fce30291ae2b9dd40d6bd2b4308d4326736" Sep 10 03:00:03.640778 ip-10-0-133-37 kubenswrapper[2560]: E0910 03:00:03.640500 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2_llm(e4195eb0-a460-48af-946a-7657e36ea1f6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" podUID="e4195eb0-a460-48af-946a-7657e36ea1f6" Sep 10 03:00:03.848814 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:00:03.848722 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816820-q8xx9" event={"ID":"5006779d-f38f-4cce-be70-9a1af6d7d8a3","Type":"ContainerStarted","Data":"6cd18438899b5132588d05e4541628f3202fc96d370383df68348bedadd266c1"} Sep 10 03:00:03.863863 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:00:03.863809 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816820-q8xx9" podStartSLOduration=1.146608838 podStartE2EDuration="3.863793136s" totalImagesPullingTime="2.717184298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:00:03.861573997 +0000 UTC m=+662.708403278" watchObservedRunningTime="2026-09-10 03:00:03.863793136 +0000 UTC m=+662.710622403" Sep 10 03:00:04.855155 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:00:04.855120 2560 generic.go:356] "Generic (PLEG): container finished" podID="5006779d-f38f-4cce-be70-9a1af6d7d8a3" containerID="6cd18438899b5132588d05e4541628f3202fc96d370383df68348bedadd266c1" exitCode=0 Sep 10 03:00:04.855597 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:00:04.855164 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816820-q8xx9" event={"ID":"5006779d-f38f-4cce-be70-9a1af6d7d8a3","Type":"ContainerDied","Data":"6cd18438899b5132588d05e4541628f3202fc96d370383df68348bedadd266c1"} Sep 10 03:00:06.002327 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:00:06.002295 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816820-q8xx9" Sep 10 03:00:06.868909 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:00:06.868879 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816820-q8xx9" Sep 10 03:00:06.868909 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:00:06.868893 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816820-q8xx9" event={"ID":"5006779d-f38f-4cce-be70-9a1af6d7d8a3","Type":"ContainerDied","Data":"0b3f29116a2b82adbb5be7c9a76774cf14b4c1a2a3511081df7a8852fb023eec"} Sep 10 03:00:06.869125 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:00:06.868946 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0b3f29116a2b82adbb5be7c9a76774cf14b4c1a2a3511081df7a8852fb023eec" Sep 10 03:00:16.640359 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:00:16.640322 2560 scope.go:117] "RemoveContainer" containerID="d19371a202fc551b2b41b95e24737fce30291ae2b9dd40d6bd2b4308d4326736" Sep 10 03:00:19.938404 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:00:19.938364 2560 generic.go:356] "Generic (PLEG): container finished" podID="e4195eb0-a460-48af-946a-7657e36ea1f6" containerID="664fe3f9dac5c6e560ba5c8ac2d8b541fd7974f64ebb03c645433e9f4ceac944" exitCode=0 Sep 10 03:00:19.938846 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:00:19.938433 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" event={"ID":"e4195eb0-a460-48af-946a-7657e36ea1f6","Type":"ContainerDied","Data":"664fe3f9dac5c6e560ba5c8ac2d8b541fd7974f64ebb03c645433e9f4ceac944"} Sep 10 03:00:19.938846 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:00:19.938498 2560 scope.go:117] "RemoveContainer" containerID="d19371a202fc551b2b41b95e24737fce30291ae2b9dd40d6bd2b4308d4326736" Sep 10 03:00:19.938977 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:00:19.938910 2560 scope.go:117] "RemoveContainer" containerID="664fe3f9dac5c6e560ba5c8ac2d8b541fd7974f64ebb03c645433e9f4ceac944" Sep 10 03:00:19.939156 ip-10-0-133-37 kubenswrapper[2560]: E0910 03:00:19.939131 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2_llm(e4195eb0-a460-48af-946a-7657e36ea1f6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" podUID="e4195eb0-a460-48af-946a-7657e36ea1f6" Sep 10 03:00:21.985219 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:00:21.985189 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 03:00:21.985593 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:00:21.985253 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 03:00:21.985593 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:00:21.985272 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 03:00:21.985593 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:00:21.985286 2560 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 03:00:21.985731 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:00:21.985716 2560 scope.go:117] "RemoveContainer" containerID="664fe3f9dac5c6e560ba5c8ac2d8b541fd7974f64ebb03c645433e9f4ceac944" Sep 10 03:00:21.985918 ip-10-0-133-37 kubenswrapper[2560]: E0910 03:00:21.985900 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2_llm(e4195eb0-a460-48af-946a-7657e36ea1f6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" podUID="e4195eb0-a460-48af-946a-7657e36ea1f6" Sep 10 03:00:32.640649 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:00:32.640611 2560 scope.go:117] "RemoveContainer" containerID="664fe3f9dac5c6e560ba5c8ac2d8b541fd7974f64ebb03c645433e9f4ceac944" Sep 10 03:00:32.641085 ip-10-0-133-37 kubenswrapper[2560]: E0910 03:00:32.640843 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2_llm(e4195eb0-a460-48af-946a-7657e36ea1f6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" podUID="e4195eb0-a460-48af-946a-7657e36ea1f6" Sep 10 03:00:44.643400 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:00:44.641038 2560 scope.go:117] "RemoveContainer" containerID="664fe3f9dac5c6e560ba5c8ac2d8b541fd7974f64ebb03c645433e9f4ceac944" Sep 10 03:00:44.643400 ip-10-0-133-37 kubenswrapper[2560]: E0910 03:00:44.641415 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2_llm(e4195eb0-a460-48af-946a-7657e36ea1f6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" podUID="e4195eb0-a460-48af-946a-7657e36ea1f6" Sep 10 03:00:59.640175 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:00:59.640100 2560 scope.go:117] "RemoveContainer" containerID="664fe3f9dac5c6e560ba5c8ac2d8b541fd7974f64ebb03c645433e9f4ceac944" Sep 10 03:01:00.142025 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:01:00.141941 2560 generic.go:356] "Generic (PLEG): container finished" podID="e4195eb0-a460-48af-946a-7657e36ea1f6" containerID="7482bc00f47b130c09cd6fda56377f1d9c19bd839f8e3910acc8f18b28af79f8" exitCode=0 Sep 10 03:01:00.142172 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:01:00.142013 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" event={"ID":"e4195eb0-a460-48af-946a-7657e36ea1f6","Type":"ContainerDied","Data":"7482bc00f47b130c09cd6fda56377f1d9c19bd839f8e3910acc8f18b28af79f8"} Sep 10 03:01:00.142172 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:01:00.142075 2560 scope.go:117] "RemoveContainer" containerID="664fe3f9dac5c6e560ba5c8ac2d8b541fd7974f64ebb03c645433e9f4ceac944" Sep 10 03:01:00.142434 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:01:00.142418 2560 scope.go:117] "RemoveContainer" containerID="7482bc00f47b130c09cd6fda56377f1d9c19bd839f8e3910acc8f18b28af79f8" Sep 10 03:01:00.142696 ip-10-0-133-37 kubenswrapper[2560]: E0910 03:01:00.142668 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2_llm(e4195eb0-a460-48af-946a-7657e36ea1f6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" podUID="e4195eb0-a460-48af-946a-7657e36ea1f6" Sep 10 03:01:01.985872 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:01:01.985843 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 03:01:01.986250 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:01:01.985903 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 03:01:01.986250 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:01:01.985919 2560 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 03:01:01.986250 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:01:01.985931 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 03:01:01.986471 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:01:01.986276 2560 scope.go:117] "RemoveContainer" containerID="7482bc00f47b130c09cd6fda56377f1d9c19bd839f8e3910acc8f18b28af79f8" Sep 10 03:01:01.986471 ip-10-0-133-37 kubenswrapper[2560]: E0910 03:01:01.986459 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2_llm(e4195eb0-a460-48af-946a-7657e36ea1f6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" podUID="e4195eb0-a460-48af-946a-7657e36ea1f6" Sep 10 03:01:15.640323 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:01:15.640282 2560 scope.go:117] "RemoveContainer" containerID="7482bc00f47b130c09cd6fda56377f1d9c19bd839f8e3910acc8f18b28af79f8" Sep 10 03:01:15.640797 ip-10-0-133-37 kubenswrapper[2560]: E0910 03:01:15.640476 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2_llm(e4195eb0-a460-48af-946a-7657e36ea1f6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" podUID="e4195eb0-a460-48af-946a-7657e36ea1f6" Sep 10 03:01:30.640006 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:01:30.639978 2560 scope.go:117] "RemoveContainer" containerID="7482bc00f47b130c09cd6fda56377f1d9c19bd839f8e3910acc8f18b28af79f8" Sep 10 03:01:30.640421 ip-10-0-133-37 kubenswrapper[2560]: E0910 03:01:30.640164 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2_llm(e4195eb0-a460-48af-946a-7657e36ea1f6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" podUID="e4195eb0-a460-48af-946a-7657e36ea1f6" Sep 10 03:01:45.641140 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:01:45.641111 2560 scope.go:117] "RemoveContainer" containerID="7482bc00f47b130c09cd6fda56377f1d9c19bd839f8e3910acc8f18b28af79f8" Sep 10 03:01:45.641545 ip-10-0-133-37 kubenswrapper[2560]: E0910 03:01:45.641278 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2_llm(e4195eb0-a460-48af-946a-7657e36ea1f6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" podUID="e4195eb0-a460-48af-946a-7657e36ea1f6" Sep 10 03:01:59.640837 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:01:59.640798 2560 scope.go:117] "RemoveContainer" containerID="7482bc00f47b130c09cd6fda56377f1d9c19bd839f8e3910acc8f18b28af79f8" Sep 10 03:01:59.641401 ip-10-0-133-37 kubenswrapper[2560]: E0910 03:01:59.641048 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2_llm(e4195eb0-a460-48af-946a-7657e36ea1f6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" podUID="e4195eb0-a460-48af-946a-7657e36ea1f6" Sep 10 03:02:10.639987 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:02:10.639954 2560 scope.go:117] "RemoveContainer" containerID="7482bc00f47b130c09cd6fda56377f1d9c19bd839f8e3910acc8f18b28af79f8" Sep 10 03:02:10.640379 ip-10-0-133-37 kubenswrapper[2560]: E0910 03:02:10.640138 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2_llm(e4195eb0-a460-48af-946a-7657e36ea1f6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" podUID="e4195eb0-a460-48af-946a-7657e36ea1f6" Sep 10 03:02:22.641094 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:02:22.641009 2560 scope.go:117] "RemoveContainer" containerID="7482bc00f47b130c09cd6fda56377f1d9c19bd839f8e3910acc8f18b28af79f8" Sep 10 03:02:23.553546 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:02:23.553493 2560 generic.go:356] "Generic (PLEG): container finished" podID="e4195eb0-a460-48af-946a-7657e36ea1f6" containerID="46346a3fb28810d44b2166aa2b0eff26e0e323cd03dca76b1f978442faea2d8a" exitCode=0 Sep 10 03:02:23.553714 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:02:23.553544 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" event={"ID":"e4195eb0-a460-48af-946a-7657e36ea1f6","Type":"ContainerDied","Data":"46346a3fb28810d44b2166aa2b0eff26e0e323cd03dca76b1f978442faea2d8a"} Sep 10 03:02:23.553714 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:02:23.553630 2560 scope.go:117] "RemoveContainer" containerID="7482bc00f47b130c09cd6fda56377f1d9c19bd839f8e3910acc8f18b28af79f8" Sep 10 03:02:23.554003 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:02:23.553983 2560 scope.go:117] "RemoveContainer" containerID="46346a3fb28810d44b2166aa2b0eff26e0e323cd03dca76b1f978442faea2d8a" Sep 10 03:02:23.554182 ip-10-0-133-37 kubenswrapper[2560]: E0910 03:02:23.554163 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2_llm(e4195eb0-a460-48af-946a-7657e36ea1f6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" podUID="e4195eb0-a460-48af-946a-7657e36ea1f6" Sep 10 03:02:31.985764 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:02:31.985728 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 03:02:31.986231 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:02:31.985807 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 03:02:31.986231 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:02:31.985836 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 03:02:31.986231 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:02:31.985859 2560 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" Sep 10 03:02:31.986361 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:02:31.986243 2560 scope.go:117] "RemoveContainer" containerID="46346a3fb28810d44b2166aa2b0eff26e0e323cd03dca76b1f978442faea2d8a" Sep 10 03:02:31.986452 ip-10-0-133-37 kubenswrapper[2560]: E0910 03:02:31.986433 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2_llm(e4195eb0-a460-48af-946a-7657e36ea1f6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" podUID="e4195eb0-a460-48af-946a-7657e36ea1f6" Sep 10 03:02:43.641202 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:02:43.641151 2560 scope.go:117] "RemoveContainer" containerID="46346a3fb28810d44b2166aa2b0eff26e0e323cd03dca76b1f978442faea2d8a" Sep 10 03:02:43.641623 ip-10-0-133-37 kubenswrapper[2560]: E0910 03:02:43.641419 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2_llm(e4195eb0-a460-48af-946a-7657e36ea1f6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" podUID="e4195eb0-a460-48af-946a-7657e36ea1f6" Sep 10 03:02:56.640160 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:02:56.640126 2560 scope.go:117] "RemoveContainer" containerID="46346a3fb28810d44b2166aa2b0eff26e0e323cd03dca76b1f978442faea2d8a" Sep 10 03:02:56.640595 ip-10-0-133-37 kubenswrapper[2560]: E0910 03:02:56.640323 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2_llm(e4195eb0-a460-48af-946a-7657e36ea1f6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" podUID="e4195eb0-a460-48af-946a-7657e36ea1f6" Sep 10 03:03:08.640894 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:08.640856 2560 scope.go:117] "RemoveContainer" containerID="46346a3fb28810d44b2166aa2b0eff26e0e323cd03dca76b1f978442faea2d8a" Sep 10 03:03:08.641351 ip-10-0-133-37 kubenswrapper[2560]: E0910 03:03:08.641029 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2_llm(e4195eb0-a460-48af-946a-7657e36ea1f6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" podUID="e4195eb0-a460-48af-946a-7657e36ea1f6" Sep 10 03:03:17.046501 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:17.046467 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2_e4195eb0-a460-48af-946a-7657e36ea1f6/main/5.log" Sep 10 03:03:17.169940 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:17.169915 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2_e4195eb0-a460-48af-946a-7657e36ea1f6/main/5.log" Sep 10 03:03:20.640840 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:20.640805 2560 scope.go:117] "RemoveContainer" containerID="46346a3fb28810d44b2166aa2b0eff26e0e323cd03dca76b1f978442faea2d8a" Sep 10 03:03:20.641240 ip-10-0-133-37 kubenswrapper[2560]: E0910 03:03:20.640998 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2_llm(e4195eb0-a460-48af-946a-7657e36ea1f6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" podUID="e4195eb0-a460-48af-946a-7657e36ea1f6" Sep 10 03:03:24.080191 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:24.080154 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-64885d587b-qjgxn_286a67b5-e0ad-4452-afdd-9e3443686520/manager/0.log" Sep 10 03:03:24.552522 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:24.552479 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78f567c59b-bd7km_a18dd49f-0a00-4a30-b4c4-ed8facc0afdc/copy-manifests/0.log" Sep 10 03:03:24.558648 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:24.558624 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78f567c59b-bd7km_a18dd49f-0a00-4a30-b4c4-ed8facc0afdc/manager/0.log" Sep 10 03:03:25.156032 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:25.156004 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29816820-q8xx9_5006779d-f38f-4cce-be70-9a1af6d7d8a3/cleanup/0.log" Sep 10 03:03:25.274734 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:25.274694 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-tx5pt_e57b24ec-2ac3-422a-a51d-59c6eb668703/postgres/0.log" Sep 10 03:03:27.217857 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:27.217822 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c_d25f3ad4-85b1-4af2-b985-f2540349cda0/istio-proxy/0.log" Sep 10 03:03:27.351019 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:27.350988 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-rvcrj_11a1299c-7c3b-4f2b-8a77-6decfb51dbbb/discovery/0.log" Sep 10 03:03:27.843069 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:27.843040 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-vzvn2_15783e7b-43bb-4ee1-a4fe-bb71e22d7a69/payload-pre-processing/0.log" Sep 10 03:03:27.958155 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:27.958124 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-fh5st_d268b46e-913a-42c9-a7a5-816c01c61848/payload-processing/0.log" Sep 10 03:03:28.072042 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:28.072011 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-699696b566-qzknj_9733272a-e52f-47ca-abc7-fcbb800ec915/router/0.log" Sep 10 03:03:28.664765 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:28.664738 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-66x7x_fba4f586-87c6-41ab-a94e-a50ffce60176/main/0.log" Sep 10 03:03:29.005203 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:29.005112 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2_e4195eb0-a460-48af-946a-7657e36ea1f6/main/5.log" Sep 10 03:03:34.640879 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:34.640848 2560 scope.go:117] "RemoveContainer" containerID="46346a3fb28810d44b2166aa2b0eff26e0e323cd03dca76b1f978442faea2d8a" Sep 10 03:03:34.641252 ip-10-0-133-37 kubenswrapper[2560]: E0910 03:03:34.641035 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2_llm(e4195eb0-a460-48af-946a-7657e36ea1f6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" podUID="e4195eb0-a460-48af-946a-7657e36ea1f6" Sep 10 03:03:37.565297 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:37.565264 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-sdxng_9ff1e29e-7033-490c-9316-def0076b8f7e/global-pull-secret-syncer/0.log" Sep 10 03:03:37.665635 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:37.665600 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-z9d9b_27be8319-a821-4e27-9dc8-411257676cb7/konnectivity-agent/0.log" Sep 10 03:03:37.687893 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:37.687834 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-37.ec2.internal_480faa79f347b524a5131c7d80428e3a/haproxy/0.log" Sep 10 03:03:43.748944 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:43.748909 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-7xqrx_4d65d823-e75e-4f53-b9f1-c6a5ed728133/cluster-monitoring-operator/0.log" Sep 10 03:03:43.900716 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:43.900682 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-sq2zn_99fb4a2a-f696-4312-8cc5-8123dfe4fc42/monitoring-plugin/0.log" Sep 10 03:03:44.009342 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:44.009244 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dnst6_478b59a6-7c53-46ed-9a1d-944981dd4b7c/node-exporter/0.log" Sep 10 03:03:44.031616 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:44.031582 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dnst6_478b59a6-7c53-46ed-9a1d-944981dd4b7c/kube-rbac-proxy/0.log" Sep 10 03:03:44.051477 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:44.051453 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dnst6_478b59a6-7c53-46ed-9a1d-944981dd4b7c/init-textfile/0.log" Sep 10 03:03:44.257776 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:44.257729 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4/prometheus/0.log" Sep 10 03:03:44.278966 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:44.278880 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4/config-reloader/0.log" Sep 10 03:03:44.302795 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:44.302770 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4/thanos-sidecar/0.log" Sep 10 03:03:44.324856 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:44.324837 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4/kube-rbac-proxy-web/0.log" Sep 10 03:03:44.345597 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:44.345560 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4/kube-rbac-proxy/0.log" Sep 10 03:03:44.375769 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:44.375742 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4/kube-rbac-proxy-thanos/0.log" Sep 10 03:03:44.397243 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:44.397218 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_297e9b2d-4bb2-456f-a1ba-fd95a63ba8a4/init-config-reloader/0.log" Sep 10 03:03:44.493463 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:44.493430 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-6ns6j_8bfcfbe2-aea8-4f6e-85cc-ad0f431adfaf/prometheus-operator-admission-webhook/0.log" Sep 10 03:03:44.528339 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:44.528291 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5dbb887c79-cbtkd_34bee801-1833-4b3f-a078-7d1e798d9c94/telemeter-client/0.log" Sep 10 03:03:44.552687 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:44.552602 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5dbb887c79-cbtkd_34bee801-1833-4b3f-a078-7d1e798d9c94/reload/0.log" Sep 10 03:03:44.572567 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:44.572536 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5dbb887c79-cbtkd_34bee801-1833-4b3f-a078-7d1e798d9c94/kube-rbac-proxy/0.log" Sep 10 03:03:44.609866 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:44.609837 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84ddd6465c-w8zvq_4eb29f8f-bee4-445f-ac27-85be89b303a1/thanos-query/0.log" Sep 10 03:03:44.634128 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:44.634101 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84ddd6465c-w8zvq_4eb29f8f-bee4-445f-ac27-85be89b303a1/kube-rbac-proxy-web/0.log" Sep 10 03:03:44.655422 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:44.655384 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84ddd6465c-w8zvq_4eb29f8f-bee4-445f-ac27-85be89b303a1/kube-rbac-proxy/0.log" Sep 10 03:03:44.680839 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:44.680805 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84ddd6465c-w8zvq_4eb29f8f-bee4-445f-ac27-85be89b303a1/prom-label-proxy/0.log" Sep 10 03:03:44.704817 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:44.704792 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84ddd6465c-w8zvq_4eb29f8f-bee4-445f-ac27-85be89b303a1/kube-rbac-proxy-rules/0.log" Sep 10 03:03:44.741028 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:44.740999 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84ddd6465c-w8zvq_4eb29f8f-bee4-445f-ac27-85be89b303a1/kube-rbac-proxy-metrics/0.log" Sep 10 03:03:46.981278 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:46.981253 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ghs46_a5210aa8-4171-4cc3-b65b-6237a9aee1d6/console-operator/2.log" Sep 10 03:03:46.989752 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:46.989722 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ghs46_a5210aa8-4171-4cc3-b65b-6237a9aee1d6/console-operator/3.log" Sep 10 03:03:47.605058 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:47.605025 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-nbnc9_c5d95d1c-394b-4750-99c5-ff8ea4afefd1/download-server/0.log" Sep 10 03:03:48.261602 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:48.261567 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-w9ph8_c2f7d2cb-e085-459b-904e-773fa7dad4e7/volume-data-source-validator/0.log" Sep 10 03:03:48.529139 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:48.529051 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-p4lv7/perf-node-gather-daemonset-h992b"] Sep 10 03:03:48.529571 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:48.529553 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5006779d-f38f-4cce-be70-9a1af6d7d8a3" containerName="cleanup" Sep 10 03:03:48.529673 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:48.529573 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5006779d-f38f-4cce-be70-9a1af6d7d8a3" containerName="cleanup" Sep 10 03:03:48.529729 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:48.529673 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5006779d-f38f-4cce-be70-9a1af6d7d8a3" containerName="cleanup" Sep 10 03:03:48.531950 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:48.531930 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-p4lv7/perf-node-gather-daemonset-h992b" Sep 10 03:03:48.534094 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:48.534075 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-p4lv7\"/\"openshift-service-ca.crt\"" Sep 10 03:03:48.534878 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:48.534865 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-p4lv7\"/\"kube-root-ca.crt\"" Sep 10 03:03:48.534932 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:48.534887 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-p4lv7\"/\"default-dockercfg-lqv22\"" Sep 10 03:03:48.541628 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:48.541601 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-p4lv7/perf-node-gather-daemonset-h992b"] Sep 10 03:03:48.626068 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:48.626044 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/45c6463b-63e8-4a95-8ced-3fbc5b99894e-proc\") pod \"perf-node-gather-daemonset-h992b\" (UID: \"45c6463b-63e8-4a95-8ced-3fbc5b99894e\") " pod="openshift-must-gather-p4lv7/perf-node-gather-daemonset-h992b" Sep 10 03:03:48.626179 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:48.626073 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/45c6463b-63e8-4a95-8ced-3fbc5b99894e-podres\") pod \"perf-node-gather-daemonset-h992b\" (UID: \"45c6463b-63e8-4a95-8ced-3fbc5b99894e\") " pod="openshift-must-gather-p4lv7/perf-node-gather-daemonset-h992b" Sep 10 03:03:48.626179 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:48.626157 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/45c6463b-63e8-4a95-8ced-3fbc5b99894e-sys\") pod \"perf-node-gather-daemonset-h992b\" (UID: \"45c6463b-63e8-4a95-8ced-3fbc5b99894e\") " pod="openshift-must-gather-p4lv7/perf-node-gather-daemonset-h992b" Sep 10 03:03:48.626261 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:48.626198 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xb6f\" (UniqueName: \"kubernetes.io/projected/45c6463b-63e8-4a95-8ced-3fbc5b99894e-kube-api-access-5xb6f\") pod \"perf-node-gather-daemonset-h992b\" (UID: \"45c6463b-63e8-4a95-8ced-3fbc5b99894e\") " pod="openshift-must-gather-p4lv7/perf-node-gather-daemonset-h992b" Sep 10 03:03:48.626261 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:48.626235 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/45c6463b-63e8-4a95-8ced-3fbc5b99894e-lib-modules\") pod \"perf-node-gather-daemonset-h992b\" (UID: \"45c6463b-63e8-4a95-8ced-3fbc5b99894e\") " pod="openshift-must-gather-p4lv7/perf-node-gather-daemonset-h992b" Sep 10 03:03:48.727564 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:48.727534 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/45c6463b-63e8-4a95-8ced-3fbc5b99894e-sys\") pod \"perf-node-gather-daemonset-h992b\" (UID: \"45c6463b-63e8-4a95-8ced-3fbc5b99894e\") " pod="openshift-must-gather-p4lv7/perf-node-gather-daemonset-h992b" Sep 10 03:03:48.727931 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:48.727581 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xb6f\" (UniqueName: \"kubernetes.io/projected/45c6463b-63e8-4a95-8ced-3fbc5b99894e-kube-api-access-5xb6f\") pod \"perf-node-gather-daemonset-h992b\" (UID: \"45c6463b-63e8-4a95-8ced-3fbc5b99894e\") " pod="openshift-must-gather-p4lv7/perf-node-gather-daemonset-h992b" Sep 10 03:03:48.727931 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:48.727624 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/45c6463b-63e8-4a95-8ced-3fbc5b99894e-lib-modules\") pod \"perf-node-gather-daemonset-h992b\" (UID: \"45c6463b-63e8-4a95-8ced-3fbc5b99894e\") " pod="openshift-must-gather-p4lv7/perf-node-gather-daemonset-h992b" Sep 10 03:03:48.727931 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:48.727644 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/45c6463b-63e8-4a95-8ced-3fbc5b99894e-sys\") pod \"perf-node-gather-daemonset-h992b\" (UID: \"45c6463b-63e8-4a95-8ced-3fbc5b99894e\") " pod="openshift-must-gather-p4lv7/perf-node-gather-daemonset-h992b" Sep 10 03:03:48.727931 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:48.727670 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/45c6463b-63e8-4a95-8ced-3fbc5b99894e-proc\") pod \"perf-node-gather-daemonset-h992b\" (UID: \"45c6463b-63e8-4a95-8ced-3fbc5b99894e\") " pod="openshift-must-gather-p4lv7/perf-node-gather-daemonset-h992b" Sep 10 03:03:48.727931 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:48.727691 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/45c6463b-63e8-4a95-8ced-3fbc5b99894e-podres\") pod \"perf-node-gather-daemonset-h992b\" (UID: \"45c6463b-63e8-4a95-8ced-3fbc5b99894e\") " pod="openshift-must-gather-p4lv7/perf-node-gather-daemonset-h992b" Sep 10 03:03:48.727931 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:48.727748 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/45c6463b-63e8-4a95-8ced-3fbc5b99894e-proc\") pod \"perf-node-gather-daemonset-h992b\" (UID: \"45c6463b-63e8-4a95-8ced-3fbc5b99894e\") " pod="openshift-must-gather-p4lv7/perf-node-gather-daemonset-h992b" Sep 10 03:03:48.727931 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:48.727797 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/45c6463b-63e8-4a95-8ced-3fbc5b99894e-podres\") pod \"perf-node-gather-daemonset-h992b\" (UID: \"45c6463b-63e8-4a95-8ced-3fbc5b99894e\") " pod="openshift-must-gather-p4lv7/perf-node-gather-daemonset-h992b" Sep 10 03:03:48.727931 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:48.727800 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/45c6463b-63e8-4a95-8ced-3fbc5b99894e-lib-modules\") pod \"perf-node-gather-daemonset-h992b\" (UID: \"45c6463b-63e8-4a95-8ced-3fbc5b99894e\") " pod="openshift-must-gather-p4lv7/perf-node-gather-daemonset-h992b" Sep 10 03:03:48.735925 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:48.735891 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xb6f\" (UniqueName: \"kubernetes.io/projected/45c6463b-63e8-4a95-8ced-3fbc5b99894e-kube-api-access-5xb6f\") pod \"perf-node-gather-daemonset-h992b\" (UID: \"45c6463b-63e8-4a95-8ced-3fbc5b99894e\") " pod="openshift-must-gather-p4lv7/perf-node-gather-daemonset-h992b" Sep 10 03:03:48.871099 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:48.871065 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-p4lv7/perf-node-gather-daemonset-h992b" Sep 10 03:03:49.141313 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:49.141290 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-p4lv7/perf-node-gather-daemonset-h992b"] Sep 10 03:03:49.143315 ip-10-0-133-37 kubenswrapper[2560]: W0910 03:03:49.143289 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod45c6463b_63e8_4a95_8ced_3fbc5b99894e.slice/crio-155dacf5f1957022d56cff5a188e904554610394895d674f2a89124e6793fcd1 WatchSource:0}: Error finding container 155dacf5f1957022d56cff5a188e904554610394895d674f2a89124e6793fcd1: Status 404 returned error can't find the container with id 155dacf5f1957022d56cff5a188e904554610394895d674f2a89124e6793fcd1 Sep 10 03:03:49.497571 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:49.497479 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2s6nr_98643336-46ce-4f17-acb2-d8110a5bfb84/dns/0.log" Sep 10 03:03:49.523724 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:49.523685 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2s6nr_98643336-46ce-4f17-acb2-d8110a5bfb84/kube-rbac-proxy/0.log" Sep 10 03:03:49.640836 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:49.640804 2560 scope.go:117] "RemoveContainer" containerID="46346a3fb28810d44b2166aa2b0eff26e0e323cd03dca76b1f978442faea2d8a" Sep 10 03:03:49.641044 ip-10-0-133-37 kubenswrapper[2560]: E0910 03:03:49.641004 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2_llm(e4195eb0-a460-48af-946a-7657e36ea1f6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" podUID="e4195eb0-a460-48af-946a-7657e36ea1f6" Sep 10 03:03:49.666551 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:49.666501 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kkzdh_6fdc2579-93fb-494d-b179-46cf1db9aaa8/dns-node-resolver/0.log" Sep 10 03:03:49.982031 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:49.981995 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-p4lv7/perf-node-gather-daemonset-h992b" event={"ID":"45c6463b-63e8-4a95-8ced-3fbc5b99894e","Type":"ContainerStarted","Data":"259b765e4a910124f70e11d83ba6b2a9c1210265d9cb3f09468dbcedae5924ed"} Sep 10 03:03:49.982189 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:49.982081 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-p4lv7/perf-node-gather-daemonset-h992b" event={"ID":"45c6463b-63e8-4a95-8ced-3fbc5b99894e","Type":"ContainerStarted","Data":"155dacf5f1957022d56cff5a188e904554610394895d674f2a89124e6793fcd1"} Sep 10 03:03:49.982189 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:49.982110 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-p4lv7/perf-node-gather-daemonset-h992b" Sep 10 03:03:49.996280 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:49.996233 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-p4lv7/perf-node-gather-daemonset-h992b" podStartSLOduration=1.996220012 podStartE2EDuration="1.996220012s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 03:03:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:03:49.995456432 +0000 UTC m=+888.842285704" watchObservedRunningTime="2026-09-10 03:03:49.996220012 +0000 UTC m=+888.843049279" Sep 10 03:03:50.293126 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:50.293047 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mnpps_936410ac-13a7-43ca-a51f-b0bd96649ff6/node-ca/0.log" Sep 10 03:03:51.207285 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:51.207254 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c_d25f3ad4-85b1-4af2-b985-f2540349cda0/istio-proxy/0.log" Sep 10 03:03:51.271055 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:51.271021 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-rvcrj_11a1299c-7c3b-4f2b-8a77-6decfb51dbbb/discovery/0.log" Sep 10 03:03:51.369461 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:51.369429 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-vzvn2_15783e7b-43bb-4ee1-a4fe-bb71e22d7a69/payload-pre-processing/0.log" Sep 10 03:03:51.393108 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:51.393076 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-fh5st_d268b46e-913a-42c9-a7a5-816c01c61848/payload-processing/0.log" Sep 10 03:03:51.417129 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:51.417103 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-699696b566-qzknj_9733272a-e52f-47ca-abc7-fcbb800ec915/router/0.log" Sep 10 03:03:52.087475 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:52.087441 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mpnm6_007e1c44-e993-4808-b999-05799e398c2f/serve-healthcheck-canary/0.log" Sep 10 03:03:52.570759 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:52.570675 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-jxcwr_2badc582-316e-4b1e-b181-760ea2abbc08/insights-operator/1.log" Sep 10 03:03:52.571208 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:52.571191 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-jxcwr_2badc582-316e-4b1e-b181-760ea2abbc08/insights-operator/0.log" Sep 10 03:03:52.617005 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:52.616974 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2wt7g_8502f598-0d8b-429b-ab5c-c23c284b2fdb/kube-rbac-proxy/0.log" Sep 10 03:03:52.645897 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:52.645869 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2wt7g_8502f598-0d8b-429b-ab5c-c23c284b2fdb/exporter/0.log" Sep 10 03:03:52.669229 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:52.669196 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2wt7g_8502f598-0d8b-429b-ab5c-c23c284b2fdb/extractor/0.log" Sep 10 03:03:54.969047 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:54.969018 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-64885d587b-qjgxn_286a67b5-e0ad-4452-afdd-9e3443686520/manager/0.log" Sep 10 03:03:55.117447 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:55.117421 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78f567c59b-bd7km_a18dd49f-0a00-4a30-b4c4-ed8facc0afdc/manager/0.log" Sep 10 03:03:55.137212 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:55.137176 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78f567c59b-bd7km_a18dd49f-0a00-4a30-b4c4-ed8facc0afdc/copy-manifests/0.log" Sep 10 03:03:55.997955 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:55.997927 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-p4lv7/perf-node-gather-daemonset-h992b" Sep 10 03:03:56.322952 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:56.322867 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-79bc984b7c-nc22k_e1791a46-f213-4ad8-abc1-3358cb12b25d/manager/0.log" Sep 10 03:03:57.916837 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:57.916807 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c_d25f3ad4-85b1-4af2-b985-f2540349cda0/istio-proxy/0.log" Sep 10 03:03:57.967775 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:57.967747 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-rvcrj_11a1299c-7c3b-4f2b-8a77-6decfb51dbbb/discovery/0.log" Sep 10 03:03:58.039526 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:58.039483 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-vzvn2_15783e7b-43bb-4ee1-a4fe-bb71e22d7a69/payload-pre-processing/0.log" Sep 10 03:03:58.056406 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:58.056382 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-fh5st_d268b46e-913a-42c9-a7a5-816c01c61848/payload-processing/0.log" Sep 10 03:03:58.075088 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:58.075067 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-699696b566-qzknj_9733272a-e52f-47ca-abc7-fcbb800ec915/router/0.log" Sep 10 03:03:59.259688 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:59.259656 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-ggj4c_d25f3ad4-85b1-4af2-b985-f2540349cda0/istio-proxy/0.log" Sep 10 03:03:59.314870 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:59.314821 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-rvcrj_11a1299c-7c3b-4f2b-8a77-6decfb51dbbb/discovery/0.log" Sep 10 03:03:59.374182 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:59.374158 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-vzvn2_15783e7b-43bb-4ee1-a4fe-bb71e22d7a69/payload-pre-processing/0.log" Sep 10 03:03:59.391315 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:59.391286 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-fh5st_d268b46e-913a-42c9-a7a5-816c01c61848/payload-processing/0.log" Sep 10 03:03:59.410828 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:03:59.410802 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-699696b566-qzknj_9733272a-e52f-47ca-abc7-fcbb800ec915/router/0.log" Sep 10 03:04:01.603066 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:04:01.603039 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ghs46_a5210aa8-4171-4cc3-b65b-6237a9aee1d6/console-operator/2.log" Sep 10 03:04:01.604615 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:04:01.604593 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-ghs46_a5210aa8-4171-4cc3-b65b-6237a9aee1d6/console-operator/2.log" Sep 10 03:04:01.609243 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:04:01.609220 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swhbm_8a6680c4-21b8-4fe2-a763-e84858c7ef90/ovn-acl-logging/0.log" Sep 10 03:04:01.610570 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:04:01.610555 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swhbm_8a6680c4-21b8-4fe2-a763-e84858c7ef90/ovn-acl-logging/0.log" Sep 10 03:04:01.644228 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:04:01.644208 2560 scope.go:117] "RemoveContainer" containerID="46346a3fb28810d44b2166aa2b0eff26e0e323cd03dca76b1f978442faea2d8a" Sep 10 03:04:01.644439 ip-10-0-133-37 kubenswrapper[2560]: E0910 03:04:01.644419 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2_llm(e4195eb0-a460-48af-946a-7657e36ea1f6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-gw7p2" podUID="e4195eb0-a460-48af-946a-7657e36ea1f6" Sep 10 03:04:02.256518 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:04:02.256482 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-mpwt5_0a4e6681-5d69-48a7-b8e7-782f3e699b0a/kube-storage-version-migrator-operator/1.log" Sep 10 03:04:02.258147 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:04:02.258121 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-mpwt5_0a4e6681-5d69-48a7-b8e7-782f3e699b0a/kube-storage-version-migrator-operator/0.log" Sep 10 03:04:03.343986 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:04:03.343938 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-28crk_8c69b77f-74ff-4667-9c26-9beba2e52e21/kube-multus/0.log" Sep 10 03:04:03.396608 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:04:03.396580 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6d6zf_d2635452-ab19-4377-93d4-3db8d6e0aa88/kube-multus-additional-cni-plugins/0.log" Sep 10 03:04:03.418018 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:04:03.417988 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6d6zf_d2635452-ab19-4377-93d4-3db8d6e0aa88/egress-router-binary-copy/0.log" Sep 10 03:04:03.442013 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:04:03.441984 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6d6zf_d2635452-ab19-4377-93d4-3db8d6e0aa88/cni-plugins/0.log" Sep 10 03:04:03.462660 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:04:03.462636 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6d6zf_d2635452-ab19-4377-93d4-3db8d6e0aa88/bond-cni-plugin/0.log" Sep 10 03:04:03.488597 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:04:03.488568 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6d6zf_d2635452-ab19-4377-93d4-3db8d6e0aa88/routeoverride-cni/0.log" Sep 10 03:04:03.509434 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:04:03.509410 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6d6zf_d2635452-ab19-4377-93d4-3db8d6e0aa88/whereabouts-cni-bincopy/0.log" Sep 10 03:04:03.530934 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:04:03.530907 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6d6zf_d2635452-ab19-4377-93d4-3db8d6e0aa88/whereabouts-cni/0.log" Sep 10 03:04:03.893900 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:04:03.893864 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-d2sbz_c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e/network-metrics-daemon/0.log" Sep 10 03:04:03.911430 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:04:03.911386 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-d2sbz_c5f72b58-af4e-4a2a-8b7f-24270dc8ef1e/kube-rbac-proxy/0.log" Sep 10 03:04:05.218584 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:04:05.218554 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swhbm_8a6680c4-21b8-4fe2-a763-e84858c7ef90/ovn-controller/0.log" Sep 10 03:04:05.236956 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:04:05.236929 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swhbm_8a6680c4-21b8-4fe2-a763-e84858c7ef90/ovn-acl-logging/1.log" Sep 10 03:04:05.237094 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:04:05.236983 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swhbm_8a6680c4-21b8-4fe2-a763-e84858c7ef90/ovn-acl-logging/0.log" Sep 10 03:04:05.257889 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:04:05.257857 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swhbm_8a6680c4-21b8-4fe2-a763-e84858c7ef90/kube-rbac-proxy-node/0.log" Sep 10 03:04:05.279601 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:04:05.279577 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swhbm_8a6680c4-21b8-4fe2-a763-e84858c7ef90/kube-rbac-proxy-ovn-metrics/0.log" Sep 10 03:04:05.296918 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:04:05.296886 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swhbm_8a6680c4-21b8-4fe2-a763-e84858c7ef90/northd/0.log" Sep 10 03:04:05.317022 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:04:05.316994 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swhbm_8a6680c4-21b8-4fe2-a763-e84858c7ef90/nbdb/0.log" Sep 10 03:04:05.338806 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:04:05.338780 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swhbm_8a6680c4-21b8-4fe2-a763-e84858c7ef90/sbdb/0.log" Sep 10 03:04:05.393633 ip-10-0-133-37 kubenswrapper[2560]: I0910 03:04:05.393595 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-swhbm_8a6680c4-21b8-4fe2-a763-e84858c7ef90/ovnkube-controller/0.log"