Sep 10 00:48:02.188352 ip-10-0-138-204 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 10 00:48:02.188362 ip-10-0-138-204 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 10 00:48:02.188371 ip-10-0-138-204 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 10 00:48:02.188564 ip-10-0-138-204 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 10 00:48:12.420294 ip-10-0-138-204 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 10 00:48:12.420312 ip-10-0-138-204 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 85570e1c53a34f8281764c7f1a6b0828 --
Sep 10 00:49:43.373303 ip-10-0-138-204 systemd[1]: Starting Kubernetes Kubelet...
Sep 10 00:49:43.829088 ip-10-0-138-204 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 00:49:43.829088 ip-10-0-138-204 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 10 00:49:43.829088 ip-10-0-138-204 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 00:49:43.829088 ip-10-0-138-204 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 10 00:49:43.829088 ip-10-0-138-204 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 00:49:43.830076 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.829973    2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 10 00:49:43.833456 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833440    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 00:49:43.833456 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833456    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 00:49:43.833527 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833461    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 00:49:43.833527 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833466    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 00:49:43.833527 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833470    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 00:49:43.833527 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833478    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 00:49:43.833527 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833481    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 00:49:43.833527 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833484    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 00:49:43.833527 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833486    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 00:49:43.833527 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833489    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 00:49:43.833527 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833492    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 00:49:43.833527 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833494    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 00:49:43.833527 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833497    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 00:49:43.833527 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833500    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 00:49:43.833527 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833503    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 00:49:43.833527 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833505    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 00:49:43.833527 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833508    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 00:49:43.833527 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833510    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 00:49:43.833527 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833513    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 00:49:43.833527 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833516    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 00:49:43.833527 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833518    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 00:49:43.834025 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833521    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 00:49:43.834025 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833524    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 00:49:43.834025 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833526    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 00:49:43.834025 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833529    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 00:49:43.834025 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833532    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 00:49:43.834025 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833534    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 00:49:43.834025 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833537    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 00:49:43.834025 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833540    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 00:49:43.834025 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833543    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 00:49:43.834025 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833546    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 00:49:43.834025 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833549    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 00:49:43.834025 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833551    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 00:49:43.834025 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833553    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 00:49:43.834025 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833556    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 00:49:43.834025 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833559    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 00:49:43.834025 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833561    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 00:49:43.834025 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833564    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 00:49:43.834025 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833567    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 00:49:43.834025 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833569    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 00:49:43.834025 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833572    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 00:49:43.834502 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833574    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 00:49:43.834502 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833577    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 00:49:43.834502 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833579    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 00:49:43.834502 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833582    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 00:49:43.834502 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833584    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 00:49:43.834502 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833587    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 00:49:43.834502 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833589    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 00:49:43.834502 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833592    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 00:49:43.834502 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833594    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 00:49:43.834502 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833596    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 00:49:43.834502 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833599    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 00:49:43.834502 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833601    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 00:49:43.834502 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833604    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 00:49:43.834502 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833607    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 00:49:43.834502 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833610    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 00:49:43.834502 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833615    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 00:49:43.834502 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833620    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 00:49:43.834502 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833623    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 00:49:43.834502 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833626    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 00:49:43.834502 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833629    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 00:49:43.834995 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833632    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 00:49:43.834995 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833634    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 00:49:43.834995 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833637    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 00:49:43.834995 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833639    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 00:49:43.834995 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833642    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 00:49:43.834995 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833644    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 00:49:43.834995 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833647    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 00:49:43.834995 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833650    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 00:49:43.834995 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833652    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 00:49:43.834995 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833655    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 00:49:43.834995 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833658    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 00:49:43.834995 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833661    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 00:49:43.834995 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833664    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 00:49:43.834995 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833667    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 00:49:43.834995 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833669    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 00:49:43.834995 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833672    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 00:49:43.834995 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833675    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 00:49:43.834995 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833679    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 00:49:43.834995 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833681    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 00:49:43.834995 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833684    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 00:49:43.835479 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833686    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 00:49:43.835479 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833689    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 00:49:43.835479 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833691    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 00:49:43.835479 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833694    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 00:49:43.835479 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833698    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 00:49:43.835479 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833701    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 00:49:43.835479 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833703    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 00:49:43.835479 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833706    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 00:49:43.835479 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833708    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 00:49:43.835479 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833711    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 00:49:43.835479 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833714    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 00:49:43.835479 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833716    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 00:49:43.835479 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833719    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 00:49:43.835479 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833722    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 00:49:43.835479 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833724    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 00:49:43.835479 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833727    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 00:49:43.835479 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833730    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 00:49:43.835479 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.833734    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 00:49:43.836572 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836560    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 00:49:43.836572 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836572    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 00:49:43.836636 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836575    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 00:49:43.836636 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836579    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 00:49:43.836636 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836582    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 00:49:43.836636 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836585    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 00:49:43.836636 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836588    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 00:49:43.836636 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836590    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 00:49:43.836636 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836593    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 00:49:43.836636 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836596    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 00:49:43.836636 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836599    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 00:49:43.836636 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836603    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 00:49:43.836636 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836605    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 00:49:43.836636 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836608    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 00:49:43.836636 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836611    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 00:49:43.836636 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836613    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 00:49:43.836636 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836616    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 00:49:43.836636 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836619    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 00:49:43.836636 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836621    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 00:49:43.836636 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836624    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 00:49:43.836636 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836626    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 00:49:43.836636 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836629    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 00:49:43.837155 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836631    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 00:49:43.837155 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836634    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 00:49:43.837155 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836637    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 00:49:43.837155 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836640    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 00:49:43.837155 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836643    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 00:49:43.837155 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836646    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 00:49:43.837155 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836648    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 00:49:43.837155 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836651    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 00:49:43.837155 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836654    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 00:49:43.837155 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836657    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 00:49:43.837155 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836659    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 00:49:43.837155 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836662    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 00:49:43.837155 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836664    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 00:49:43.837155 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836667    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 00:49:43.837155 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836670    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 00:49:43.837155 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836672    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 00:49:43.837155 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836675    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 00:49:43.837155 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836678    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 00:49:43.837155 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836680    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 00:49:43.837155 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836683    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 00:49:43.837644 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836685    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 00:49:43.837644 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836688    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 00:49:43.837644 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836690    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 00:49:43.837644 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836693    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 00:49:43.837644 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836696    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 00:49:43.837644 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836698    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 00:49:43.837644 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836701    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 00:49:43.837644 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836704    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 00:49:43.837644 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836706    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 00:49:43.837644 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836709    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 00:49:43.837644 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836711    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 00:49:43.837644 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836714    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 00:49:43.837644 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836717    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 00:49:43.837644 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836719    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 00:49:43.837644 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836722    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 00:49:43.837644 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836724    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 00:49:43.837644 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836727    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 00:49:43.837644 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836730    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 00:49:43.837644 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836733    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 00:49:43.837644 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836735    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 00:49:43.838154 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836738    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 00:49:43.838154 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836741    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 00:49:43.838154 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836743    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 00:49:43.838154 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836746    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 00:49:43.838154 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836748    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 00:49:43.838154 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836751    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 00:49:43.838154 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836753    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 00:49:43.838154 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836756    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 00:49:43.838154 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836760    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 00:49:43.838154 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836764    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 00:49:43.838154 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836767    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 00:49:43.838154 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836769    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 00:49:43.838154 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836772    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 00:49:43.838154 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836774    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 00:49:43.838154 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836777    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 00:49:43.838154 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836780    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 00:49:43.838154 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836783    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 00:49:43.838154 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836785    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 00:49:43.838154 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836790    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 00:49:43.838639 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836794    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 00:49:43.838639 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836797    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 00:49:43.838639 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836800    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 00:49:43.838639 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836802    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 00:49:43.838639 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836805    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 00:49:43.838639 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836808    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 00:49:43.838639 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836811    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 00:49:43.838639 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836814    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 00:49:43.838639 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836816    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 00:49:43.838639 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836819    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 00:49:43.838639 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836823    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 00:49:43.838639 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836826    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 00:49:43.838639 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836829    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 00:49:43.838639 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836832    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 00:49:43.838639 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836835    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 00:49:43.838639 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836838    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 00:49:43.838639 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836840    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 00:49:43.838639 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.836843    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 00:49:43.838639 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837277    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 00:49:43.839108 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837283    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 00:49:43.839108 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837286    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 00:49:43.839108 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837290    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 00:49:43.839108 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837293    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 00:49:43.839108 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837296    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 00:49:43.839108 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837299    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 00:49:43.839108 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837302    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 00:49:43.839108 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837305    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 00:49:43.839108 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837308    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 00:49:43.839108 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837311    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 00:49:43.839108 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837314    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 00:49:43.839108 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837317    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 00:49:43.839108 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837320    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 00:49:43.839108 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837323    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 00:49:43.839108 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837326    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 00:49:43.839108 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837328    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 00:49:43.839108 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837331    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 00:49:43.839108 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837333    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 00:49:43.839108 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837337    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 00:49:43.839583 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837340    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 00:49:43.839583 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837342    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 00:49:43.839583 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837345    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 00:49:43.839583 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837347    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 00:49:43.839583 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837350    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 00:49:43.839583 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837353    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 00:49:43.839583 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837355    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 00:49:43.839583 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837358    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 00:49:43.839583 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837360    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 00:49:43.839583 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837363    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 00:49:43.839583 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837365    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 00:49:43.839583 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837368    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 00:49:43.839583 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837371    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 00:49:43.839583 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837373    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 00:49:43.839583 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837376    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 00:49:43.839583 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837378    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 00:49:43.839583 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837381    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 00:49:43.839583 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837384    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 00:49:43.839583 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837387    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 00:49:43.839583 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837389    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 00:49:43.840127 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837392    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 00:49:43.840127 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837394    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 00:49:43.840127 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837397    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 00:49:43.840127 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837400    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 00:49:43.840127 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837403    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 00:49:43.840127 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837405    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 00:49:43.840127 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837408    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 00:49:43.840127 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837410    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 00:49:43.840127 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837413    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 00:49:43.840127 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837415    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 00:49:43.840127 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837418    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 00:49:43.840127 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837423    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 00:49:43.840127 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837426    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 00:49:43.840127 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837429    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 00:49:43.840127 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837432    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 00:49:43.840127 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837434    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 00:49:43.840127 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837438    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 00:49:43.840127 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837441    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 00:49:43.840127 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837443    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 00:49:43.840683 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837446    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 00:49:43.840683 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837449    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 00:49:43.840683 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837451    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 00:49:43.840683 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837454    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 00:49:43.840683 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837457    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 00:49:43.840683 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837460    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 00:49:43.840683 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837463    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 00:49:43.840683 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837466    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 00:49:43.840683 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837469    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 00:49:43.840683 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837472    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 00:49:43.840683 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837474    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 00:49:43.840683 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837478    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 00:49:43.840683 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837480    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 00:49:43.840683 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837482    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 00:49:43.840683 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837485    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 00:49:43.840683 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837488    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 00:49:43.840683 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837492    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 00:49:43.840683 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837494    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 00:49:43.840683 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837497    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 00:49:43.841176 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837501    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 00:49:43.841176 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837504    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 00:49:43.841176 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837507    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 00:49:43.841176 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837510    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 00:49:43.841176 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837513    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 00:49:43.841176 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837515    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 00:49:43.841176 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837518    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 00:49:43.841176 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837520    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 00:49:43.841176 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837523    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 00:49:43.841176 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837525    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 00:49:43.841176 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837528    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 00:49:43.841176 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837530    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 00:49:43.841176 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837533    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 00:49:43.841176 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837535    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 00:49:43.841176 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837539    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 00:49:43.841176 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837542    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 00:49:43.841176 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837545    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 00:49:43.841176 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837547    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 00:49:43.841176 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837550    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 00:49:43.841176 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837564    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 00:49:43.841702 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.837568    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 00:49:43.841702 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837632    2563 flags.go:64] FLAG: --address="0.0.0.0"
Sep 10 00:49:43.841702 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837666    2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 10 00:49:43.841702 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837672    2563 flags.go:64] FLAG: --anonymous-auth="true"
Sep 10 00:49:43.841702 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837677    2563 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 10 00:49:43.841702 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837682    2563 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 10 00:49:43.841702 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837686    2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 10 00:49:43.841702 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837690    2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 10 00:49:43.841702 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837695    2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 10 00:49:43.841702 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837698    2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 10 00:49:43.841702 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837701    2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 10 00:49:43.841702 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837705    2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 10 00:49:43.841702 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837708    2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 10 00:49:43.841702 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837711    2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 10 00:49:43.841702 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837714    2563 flags.go:64] FLAG: --cgroup-root=""
Sep 10 00:49:43.841702 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837717    2563 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 10 00:49:43.841702 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837720    2563 flags.go:64] FLAG: --client-ca-file=""
Sep 10 00:49:43.841702 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837722    2563 flags.go:64] FLAG: --cloud-provider="external"
Sep 10 00:49:43.841702 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837725    2563 flags.go:64] FLAG: --cluster-dns="[]"
Sep 10 00:49:43.841702 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837731    2563 flags.go:64] FLAG: --cluster-domain=""
Sep 10 00:49:43.841702 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837734    2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 10 00:49:43.841702 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837737    2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 10 00:49:43.841702 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837740    2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 10 00:49:43.841702 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837743    2563 flags.go:64] FLAG: --container-log-max-files="5"
Sep 10 00:49:43.842287 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837747    2563 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 10 00:49:43.842287 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837750    2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 10 00:49:43.842287 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837753    2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 10 00:49:43.842287 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837756    2563 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 10 00:49:43.842287 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837759    2563 flags.go:64] FLAG: --contention-profiling="false"
Sep 10 00:49:43.842287 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837763    2563 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 10 00:49:43.842287 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837766    2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 10 00:49:43.842287 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837769    2563 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 10 00:49:43.842287 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837772    2563 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 10 00:49:43.842287 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837776    2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 10 00:49:43.842287 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837779    2563 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 10 00:49:43.842287 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837782    2563 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 10 00:49:43.842287 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837785    2563 flags.go:64] FLAG: --enable-load-reader="false"
Sep 10 00:49:43.842287 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837788    2563 flags.go:64] FLAG: --enable-server="true"
Sep 10 00:49:43.842287 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837792    2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 10 00:49:43.842287 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837797    2563 flags.go:64] FLAG: --event-burst="100"
Sep 10 00:49:43.842287 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837799    2563 flags.go:64] FLAG: --event-qps="50"
Sep 10 00:49:43.842287 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837802    2563 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 10 00:49:43.842287 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837805    2563 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 10 00:49:43.842287 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837808    2563 flags.go:64] FLAG: --eviction-hard=""
Sep 10 00:49:43.842287 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837812    2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 10 00:49:43.842287 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837815    2563 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 10 00:49:43.842287 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837818    2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 10 00:49:43.842287 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837821    2563 flags.go:64] FLAG: --eviction-soft=""
Sep 10 00:49:43.842287 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837824    2563 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 10 00:49:43.842905 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837827    2563 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 10 00:49:43.842905 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837830    2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 10 00:49:43.842905 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837833    2563 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 10 00:49:43.842905 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837836    2563 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 10 00:49:43.842905 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837839    2563 flags.go:64] FLAG: --fail-swap-on="true"
Sep 10 00:49:43.842905 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837842    2563 flags.go:64] FLAG: --feature-gates=""
Sep 10 00:49:43.842905 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837846    2563 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 10 00:49:43.842905 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837849    2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 10 00:49:43.842905 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837852    2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 10 00:49:43.842905 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837855    2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 10 00:49:43.842905 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837858    2563 flags.go:64] FLAG: --healthz-port="10248"
Sep 10 00:49:43.842905 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837861    2563 flags.go:64] FLAG: --help="false"
Sep 10 00:49:43.842905 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837864    2563 flags.go:64] FLAG: --hostname-override="ip-10-0-138-204.ec2.internal"
Sep 10 00:49:43.842905 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837867    2563 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 10 00:49:43.842905 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837870    2563 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 10 00:49:43.842905 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837872    2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 10 00:49:43.842905 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837876    2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 10 00:49:43.842905 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837879    2563 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 10 00:49:43.842905 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837882    2563 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 10 00:49:43.842905 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837885    2563 flags.go:64] FLAG: --image-service-endpoint=""
Sep 10 00:49:43.842905 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837888    2563 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 10 00:49:43.842905 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837891    2563 flags.go:64] FLAG: --kube-api-burst="100"
Sep 10 00:49:43.842905 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837894    2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 10 00:49:43.843507 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837898    2563 flags.go:64] FLAG: --kube-api-qps="50"
Sep 10 00:49:43.843507 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837901    2563 flags.go:64] FLAG: --kube-reserved=""
Sep 10 00:49:43.843507 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837904    2563 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 10 00:49:43.843507 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837906    2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 10 00:49:43.843507 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837910    2563 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 10 00:49:43.843507 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837912    2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 10 00:49:43.843507 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837915    2563 flags.go:64] FLAG: --lock-file=""
Sep 10 00:49:43.843507 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837918    2563 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 10 00:49:43.843507 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837920    2563 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 10 00:49:43.843507 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837923    2563 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 10 00:49:43.843507 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837929    2563 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 10 00:49:43.843507 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837932    2563 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 10 00:49:43.843507 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837934    2563 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 10 00:49:43.843507 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837937    2563 flags.go:64] FLAG: --logging-format="text"
Sep 10 00:49:43.843507 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837940    2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 10 00:49:43.843507 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837943    2563 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 10 00:49:43.843507 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837946    2563 flags.go:64] FLAG: --manifest-url=""
Sep 10 00:49:43.843507 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837949    2563 flags.go:64] FLAG: --manifest-url-header=""
Sep 10 00:49:43.843507 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837953    2563 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 10 00:49:43.843507 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837956    2563 flags.go:64] FLAG: --max-open-files="1000000"
Sep 10 00:49:43.843507 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837961    2563 flags.go:64] FLAG: --max-pods="110"
Sep 10 00:49:43.843507 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837964    2563 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 10 00:49:43.843507 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837967    2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 10 00:49:43.843507 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837970    2563 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 10 00:49:43.843507 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837972    2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 10 00:49:43.844148 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837977    2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 10 00:49:43.844148 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837993    2563 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 10 00:49:43.844148 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.837997    2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 10 00:49:43.844148 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838004    2563 flags.go:64] FLAG: --node-status-max-images="50"
Sep 10 00:49:43.844148 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838007    2563 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 10 00:49:43.844148 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838010    2563 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 10 00:49:43.844148 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838013    2563 flags.go:64] FLAG: --pod-cidr=""
Sep 10 00:49:43.844148 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838018    2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 10 00:49:43.844148 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838023    2563 flags.go:64] FLAG: --pod-manifest-path=""
Sep 10 00:49:43.844148 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838026    2563 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 10 00:49:43.844148 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838029    2563 flags.go:64] FLAG: --pods-per-core="0"
Sep 10 00:49:43.844148 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838032    2563 flags.go:64] FLAG: --port="10250"
Sep 10 00:49:43.844148 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838035    2563 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 10 00:49:43.844148 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838038    2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-003f0e405da8a2595"
Sep 10 00:49:43.844148 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838041    2563 flags.go:64] FLAG: --qos-reserved=""
Sep 10 00:49:43.844148 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838044    2563 flags.go:64] FLAG: --read-only-port="10255"
Sep 10 00:49:43.844148 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838047    2563 flags.go:64] FLAG: --register-node="true"
Sep 10 00:49:43.844148 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838050    2563 flags.go:64] FLAG: --register-with-taints=""
Sep 10 00:49:43.844148 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838053    2563 flags.go:64] FLAG: --registry-burst="10"
Sep 10 00:49:43.844148 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838056    2563 flags.go:64] FLAG: --registry-qps="5"
Sep 10 00:49:43.844148 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838059    2563 flags.go:64] FLAG: --reserved-cpus=""
Sep 10 00:49:43.844148 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838061    2563 flags.go:64] FLAG: --reserved-memory=""
Sep 10 00:49:43.844148 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838065    2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 10 00:49:43.844148 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838068    2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 10 00:49:43.844148 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838071    2563 flags.go:64] FLAG: --rotate-certificates="false"
Sep 10 00:49:43.844769 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838074    2563 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 10 00:49:43.844769 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838077    2563 flags.go:64] FLAG: --runonce="false"
Sep 10 00:49:43.844769 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838080    2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 10 00:49:43.844769 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838086    2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 10 00:49:43.844769 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838089    2563 flags.go:64] FLAG: --seccomp-default="false"
Sep 10 00:49:43.844769 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838091    2563 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 10 00:49:43.844769 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838094    2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 10 00:49:43.844769 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838099    2563 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 10 00:49:43.844769 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838103    2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 10 00:49:43.844769 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838106    2563 flags.go:64] FLAG: --storage-driver-password="root"
Sep 10 00:49:43.844769 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838109    2563 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 10 00:49:43.844769 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838112    2563 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 10 00:49:43.844769 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838115    2563 flags.go:64] FLAG: --storage-driver-user="root"
Sep 10 00:49:43.844769 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838118    2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 10 00:49:43.844769 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838121    2563 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 10 00:49:43.844769 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838124    2563 flags.go:64] FLAG: --system-cgroups=""
Sep 10 00:49:43.844769 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838127    2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 10 00:49:43.844769 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838132    2563 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 10 00:49:43.844769 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838135    2563 flags.go:64] FLAG: --tls-cert-file=""
Sep 10 00:49:43.844769 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838138    2563 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 10 00:49:43.844769 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838142    2563 flags.go:64] FLAG: --tls-min-version=""
Sep 10 00:49:43.844769 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838145    2563 flags.go:64] FLAG: --tls-private-key-file=""
Sep 10 00:49:43.844769 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838148    2563 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 10 00:49:43.844769 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838151    2563 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 10 00:49:43.844769 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838154    2563 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 10 00:49:43.845463 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838157    2563 flags.go:64] FLAG: --v="2"
Sep 10 00:49:43.845463 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838160    2563 flags.go:64] FLAG: --version="false"
Sep 10 00:49:43.845463 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838165    2563 flags.go:64] FLAG: --vmodule=""
Sep 10 00:49:43.845463 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838168    2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 10 00:49:43.845463 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838172    2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 10 00:49:43.845463 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838289    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 00:49:43.845463 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838292    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 00:49:43.845463 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838295    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 00:49:43.845463 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838298    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 00:49:43.845463 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838301    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 00:49:43.845463 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838306    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 00:49:43.845463 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838309    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 00:49:43.845463 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838312    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 00:49:43.845463 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838315    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 00:49:43.845463 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838319    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 00:49:43.845463 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838322    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 00:49:43.845463 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838325    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 00:49:43.845463 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838327    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 00:49:43.845463 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838330    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 00:49:43.845463 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838333    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 00:49:43.845463 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838336    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 00:49:43.845980 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838339    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 00:49:43.845980 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838342    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 00:49:43.845980 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838345    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 00:49:43.845980 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838347    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 00:49:43.845980 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838350    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 00:49:43.845980 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838353    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 00:49:43.845980 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838355    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 00:49:43.845980 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838357    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 00:49:43.845980 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838360    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 00:49:43.845980 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838363    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 00:49:43.845980 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838365    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 00:49:43.845980 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838368    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 00:49:43.845980 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838370    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 00:49:43.845980 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838373    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 00:49:43.845980 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838375    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 00:49:43.845980 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838378    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 00:49:43.845980 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838380    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 00:49:43.845980 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838383    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 00:49:43.845980 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838385    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 00:49:43.845980 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838388    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 00:49:43.846481 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838390    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 00:49:43.846481 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838394    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 00:49:43.846481 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838397    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 00:49:43.846481 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838399    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 00:49:43.846481 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838402    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 00:49:43.846481 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838406    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 00:49:43.846481 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838408    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 00:49:43.846481 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838411    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 00:49:43.846481 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838413    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 00:49:43.846481 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838416    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 00:49:43.846481 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838419    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 00:49:43.846481 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838421    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 00:49:43.846481 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838424    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 00:49:43.846481 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838427    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 00:49:43.846481 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838429    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 00:49:43.846481 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838433    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 00:49:43.846481 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838435    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 00:49:43.846481 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838438    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 00:49:43.846481 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838440    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 00:49:43.846481 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838443    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 00:49:43.846970 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838445    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 00:49:43.846970 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838448    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 00:49:43.846970 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838452    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 00:49:43.846970 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838455    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 00:49:43.846970 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838458    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 00:49:43.846970 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838462    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 00:49:43.846970 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838465    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 00:49:43.846970 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838468    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 00:49:43.846970 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838471    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 00:49:43.846970 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838474    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 00:49:43.846970 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838476    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 00:49:43.846970 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838479    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 00:49:43.846970 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838481    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 00:49:43.846970 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838485    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 00:49:43.846970 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838488    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 00:49:43.846970 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838490    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 00:49:43.846970 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838493    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 00:49:43.846970 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838497    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 00:49:43.846970 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838499    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 00:49:43.847448 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838502    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 00:49:43.847448 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838505    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 00:49:43.847448 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838508    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 00:49:43.847448 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838510    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 00:49:43.847448 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838513    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 00:49:43.847448 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838516    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 00:49:43.847448 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838518    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 00:49:43.847448 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838528    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 00:49:43.847448 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838531    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 00:49:43.847448 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838534    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 00:49:43.847448 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838536    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 00:49:43.847448 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838540    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 00:49:43.847448 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838544    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 00:49:43.847448 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838547    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 00:49:43.847448 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838549    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 00:49:43.847448 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838552    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 00:49:43.847448 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838555    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 00:49:43.847448 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838557    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 00:49:43.847448 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838560    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 00:49:43.848165 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838563    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 00:49:43.848165 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838565    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 00:49:43.848165 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838568    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 00:49:43.848165 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838570    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 00:49:43.848165 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.838573    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 00:49:43.848165 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.838578    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 00:49:43.848165 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.846622    2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 10 00:49:43.848165 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.846640    2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 10 00:49:43.848165 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.846656    2563 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 10 00:49:43.848165 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.846665    2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 10 00:49:43.848165 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846737    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 00:49:43.848165 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846742    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 00:49:43.848165 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846745    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 00:49:43.848165 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846748    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 00:49:43.848165 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846752    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 00:49:43.848781 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846755    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 00:49:43.848781 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846758    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 00:49:43.848781 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846761    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 00:49:43.848781 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846765    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 00:49:43.848781 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846769    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 00:49:43.848781 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846772    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 00:49:43.848781 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846774    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 00:49:43.848781 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846777    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 00:49:43.848781 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846780    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 00:49:43.848781 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846782    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 00:49:43.848781 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846785    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 00:49:43.848781 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846788    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 00:49:43.848781 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846790    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 00:49:43.848781 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846793    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 00:49:43.848781 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846796    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 00:49:43.848781 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846800    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 00:49:43.848781 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846803    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 00:49:43.848781 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846806    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 00:49:43.848781 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846809    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 00:49:43.849313 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846811    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 00:49:43.849313 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846814    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 00:49:43.849313 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846816    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 00:49:43.849313 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846819    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 00:49:43.849313 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846821    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 00:49:43.849313 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846830    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 00:49:43.849313 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846833    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 00:49:43.849313 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846836    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 00:49:43.849313 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846838    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 00:49:43.849313 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846841    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 00:49:43.849313 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846845    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 00:49:43.849313 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846848    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 00:49:43.849313 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846851    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 00:49:43.849313 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846854    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 00:49:43.849313 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846857    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 00:49:43.849313 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846860    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 00:49:43.849313 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846863    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 00:49:43.849313 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846866    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 00:49:43.849313 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846869    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 00:49:43.849884 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846872    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 00:49:43.849884 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846874    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 00:49:43.849884 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846877    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 00:49:43.849884 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846879    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 00:49:43.849884 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846882    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 00:49:43.849884 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846884    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 00:49:43.849884 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846887    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 00:49:43.849884 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846889    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 00:49:43.849884 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846891    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 00:49:43.849884 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846894    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 00:49:43.849884 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846896    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 00:49:43.849884 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846899    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 00:49:43.849884 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846902    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 00:49:43.849884 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846904    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 00:49:43.849884 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846907    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 00:49:43.849884 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846909    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 00:49:43.849884 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846912    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 00:49:43.849884 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846914    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 00:49:43.849884 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846916    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 00:49:43.849884 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846925    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 00:49:43.850395 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846928    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 00:49:43.850395 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846930    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 00:49:43.850395 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846933    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 00:49:43.850395 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846936    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 00:49:43.850395 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846938    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 00:49:43.850395 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846941    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 00:49:43.850395 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846943    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 00:49:43.850395 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846946    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 00:49:43.850395 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846954    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 00:49:43.850395 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846957    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 00:49:43.850395 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846960    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 00:49:43.850395 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846962    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 00:49:43.850395 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846965    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 00:49:43.850395 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846967    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 00:49:43.850395 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846970    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 00:49:43.850395 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846973    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 00:49:43.850395 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846976    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 00:49:43.850395 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846978    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 00:49:43.850395 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846993    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 00:49:43.850395 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846996    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 00:49:43.850871 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.846999    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 00:49:43.850871 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847001    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 00:49:43.850871 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847004    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 00:49:43.850871 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847007    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 00:49:43.850871 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847009    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 00:49:43.850871 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847012    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 00:49:43.850871 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847014    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 00:49:43.850871 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847017    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 00:49:43.850871 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847019    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 00:49:43.850871 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847022    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 00:49:43.850871 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847025    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 00:49:43.850871 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847027    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 00:49:43.850871 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847038    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 00:49:43.850871 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847041    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 00:49:43.850871 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847044    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 00:49:43.850871 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847046    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 00:49:43.851264 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.847051    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 00:49:43.851264 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847203    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 00:49:43.851264 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847207    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 00:49:43.851264 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847211    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 00:49:43.851264 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847214    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 00:49:43.851264 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847216    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 00:49:43.851264 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847219    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 00:49:43.851264 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847221    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 00:49:43.851264 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847224    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 00:49:43.851264 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847227    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 00:49:43.851264 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847229    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 00:49:43.851264 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847232    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 00:49:43.851264 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847234    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 00:49:43.851264 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847237    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 00:49:43.851264 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847239    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 00:49:43.851640 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847242    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 00:49:43.851640 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847245    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 00:49:43.851640 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847247    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 00:49:43.851640 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847250    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 00:49:43.851640 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847253    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 00:49:43.851640 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847255    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 00:49:43.851640 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847258    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 00:49:43.851640 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847260    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 00:49:43.851640 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847263    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 00:49:43.851640 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847265    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 00:49:43.851640 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847268    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 00:49:43.851640 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847270    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 00:49:43.851640 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847273    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 00:49:43.851640 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847276    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 00:49:43.851640 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847284    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 00:49:43.851640 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847287    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 00:49:43.851640 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847289    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 00:49:43.851640 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847292    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 00:49:43.851640 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847295    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 00:49:43.851640 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847297    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 00:49:43.852136 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847300    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 00:49:43.852136 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847302    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 00:49:43.852136 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847305    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 00:49:43.852136 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847307    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 00:49:43.852136 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847310    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 00:49:43.852136 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847312    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 00:49:43.852136 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847315    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 00:49:43.852136 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847317    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 00:49:43.852136 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847321    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 00:49:43.852136 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847324    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 00:49:43.852136 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847327    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 00:49:43.852136 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847330    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 00:49:43.852136 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847333    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 00:49:43.852136 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847335    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 00:49:43.852136 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847341    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 00:49:43.852136 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847344    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 00:49:43.852136 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847346    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 00:49:43.852136 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847349    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 00:49:43.852136 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847352    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 00:49:43.852646 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847356    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 00:49:43.852646 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847358    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 00:49:43.852646 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847361    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 00:49:43.852646 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847363    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 00:49:43.852646 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847366    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 00:49:43.852646 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847368    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 00:49:43.852646 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847371    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 00:49:43.852646 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847374    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 00:49:43.852646 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847383    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 00:49:43.852646 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847386    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 00:49:43.852646 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847389    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 00:49:43.852646 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847391    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 00:49:43.852646 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847394    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 00:49:43.852646 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847396    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 00:49:43.852646 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847399    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 00:49:43.852646 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847401    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 00:49:43.852646 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847405    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 00:49:43.852646 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847408    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 00:49:43.852646 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847411    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 00:49:43.853266 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847414    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 00:49:43.853266 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847417    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 00:49:43.853266 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847420    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 00:49:43.853266 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847422    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 00:49:43.853266 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847428    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 00:49:43.853266 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847430    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 00:49:43.853266 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847433    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 00:49:43.853266 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847435    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 00:49:43.853266 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847438    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 00:49:43.853266 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847441    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 00:49:43.853266 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847443    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 00:49:43.853266 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847446    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 00:49:43.853266 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847448    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 00:49:43.853266 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847451    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 00:49:43.853266 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847453    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 00:49:43.853266 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847456    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 00:49:43.853266 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847458    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 00:49:43.853266 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847461    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 00:49:43.853266 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847464    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 00:49:43.853266 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847466    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 00:49:43.853859 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847469    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 00:49:43.853859 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847471    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 00:49:43.853859 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847480    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 00:49:43.853859 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847482    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 00:49:43.853859 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847485    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 00:49:43.853859 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847487    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 00:49:43.853859 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:43.847490    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 00:49:43.853859 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.847495    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 00:49:43.853859 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.849715    2563 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 10 00:49:43.853859 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.852159    2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 10 00:49:43.853859 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.853269    2563 server.go:1019] "Starting client certificate rotation"
Sep 10 00:49:43.853859 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.853385    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 00:49:43.853859 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.853438    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 00:49:43.856246 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.856229    2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 00:49:43.857512 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.857494    2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 00:49:43.871295 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.871278    2563 log.go:25] "Validated CRI v1 runtime API"
Sep 10 00:49:43.874836 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.874819    2563 log.go:25] "Validated CRI v1 image API"
Sep 10 00:49:43.875671 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.875657    2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 10 00:49:43.879789 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.879770    2563 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 c8495844-9b14-4a13-80d5-2bf39ce8e4dc:/dev/nvme0n1p4 d401b4d7-d861-49bc-87fd-92fefa4491a6:/dev/nvme0n1p3]
Sep 10 00:49:43.879837 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.879788    2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 10 00:49:43.886706 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.886690    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 00:49:43.887044 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.886921    2563 manager.go:216] Machine: {Timestamp:2026-09-10 00:49:43.885396895 +0000 UTC m=+0.382696464 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3051664 MemoryCapacity:33164472320 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21ddddba838bbf139f184f6c2d1420 SystemUUID:ec21dddd-ba83-8bbf-139f-184f6c2d1420 BootID:85570e1c-53a3-4f82-8176-4c7f1a6b0828 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582234112 Type:vfs Inodes:4048397 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632894464 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:5c:c4:3c:4c:87 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:5c:c4:3c:4c:87 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6e:98:8a:58:48:f6 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164472320 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 10 00:49:43.887110 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.887046    2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 10 00:49:43.887149 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.887138    2563 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 10 00:49:43.890868 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.890833    2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 10 00:49:43.891036 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.890877    2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-204.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 10 00:49:43.891758 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.891748    2563 topology_manager.go:138] "Creating topology manager with none policy"
Sep 10 00:49:43.891793 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.891760    2563 container_manager_linux.go:309] "Creating device plugin manager"
Sep 10 00:49:43.891793 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.891768    2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 00:49:43.891793 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.891782    2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 00:49:43.893030 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.893021    2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 10 00:49:43.897657 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.897645    2563 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 00:49:43.897781 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.897772    2563 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 10 00:49:43.897825 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.897817    2563 kubelet.go:486] "Attempting to sync node with API server"
Sep 10 00:49:43.897854 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.897831    2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 10 00:49:43.897854 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.897845    2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 10 00:49:43.897910 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.897860    2563 kubelet.go:398] "Adding apiserver pod source"
Sep 10 00:49:43.897910 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.897869    2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 10 00:49:43.898293 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.898276    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 00:49:43.898293 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.898294    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 00:49:43.902038 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.902014    2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 10 00:49:43.905164 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.905148    2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 10 00:49:43.905265 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.905179    2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 10 00:49:43.906606 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.906594    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 10 00:49:43.906654 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.906610    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 10 00:49:43.906654 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.906617    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 10 00:49:43.906654 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.906622    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 10 00:49:43.906654 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.906631    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 10 00:49:43.906654 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.906636    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 10 00:49:43.906654 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.906649    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 10 00:49:43.906654 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.906654    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 10 00:49:43.906831 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.906661    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 10 00:49:43.906831 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.906667    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 10 00:49:43.906831 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.906675    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 10 00:49:43.906831 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.906680    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 10 00:49:43.907530 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.907521    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 10 00:49:43.907530 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.907529    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 10 00:49:43.911308 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.911293    2563 server.go:1268] "Started kubelet"
Sep 10 00:49:43.911447 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.911395    2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 10 00:49:43.911447 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.911415    2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 10 00:49:43.911538 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.911502    2563 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 10 00:49:43.911663 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:43.911645    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-204.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 00:49:43.911705 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.911690    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-204.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 00:49:43.911814 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:43.911795    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 00:49:43.912247 ip-10-0-138-204 systemd[1]: Started Kubernetes Kubelet.
Sep 10 00:49:43.913108 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.913089    2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 10 00:49:43.914970 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.914952    2563 server.go:310] "Adding debug handlers to kubelet server"
Sep 10 00:49:43.917164 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:43.917135    2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 10 00:49:43.919471 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:43.918298    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-204.ec2.internal.18d3cf47b6b63bb3  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-204.ec2.internal,UID:ip-10-0-138-204.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-204.ec2.internal,},FirstTimestamp:2026-09-10 00:49:43.911242675 +0000 UTC m=+0.408542245,LastTimestamp:2026-09-10 00:49:43.911242675 +0000 UTC m=+0.408542245,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-204.ec2.internal,}"
Sep 10 00:49:43.919572 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.919472    2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 10 00:49:43.919572 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.919490    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 10 00:49:43.919674 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:43.919652    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-204.ec2.internal\" not found"
Sep 10 00:49:43.920272 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.920255    2563 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 10 00:49:43.920272 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.920272    2563 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 10 00:49:43.920490 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.920259    2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 10 00:49:43.920490 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.920422    2563 reconstruct.go:97] "Volume reconstruction finished"
Sep 10 00:49:43.920490 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.920428    2563 reconciler.go:29] "Reconciler: start to sync state"
Sep 10 00:49:43.921303 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.921286    2563 factory.go:55] Registering systemd factory
Sep 10 00:49:43.921303 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.921306    2563 factory.go:223] Registration of the systemd container factory successfully
Sep 10 00:49:43.921622 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.921602    2563 factory.go:153] Registering CRI-O factory
Sep 10 00:49:43.921622 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.921623    2563 factory.go:223] Registration of the crio container factory successfully
Sep 10 00:49:43.921767 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.921674    2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 10 00:49:43.921767 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.921694    2563 factory.go:103] Registering Raw factory
Sep 10 00:49:43.921767 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.921712    2563 manager.go:1196] Started watching for new ooms in manager
Sep 10 00:49:43.922937 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.922816    2563 manager.go:349] Starting recovery of all containers
Sep 10 00:49:43.929488 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:43.929313    2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-204.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 10 00:49:43.929897 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:43.929864    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 10 00:49:43.935414 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.935395    2563 manager.go:354] Recovery completed
Sep 10 00:49:43.937575 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.937553    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hzcxr"
Sep 10 00:49:43.940721 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.940708    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 00:49:43.942883 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.942865    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-204.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 00:49:43.942973 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.942902    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-204.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 00:49:43.942973 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.942918    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-204.ec2.internal" event="NodeHasSufficientPID"
Sep 10 00:49:43.943616 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.943602    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 00:49:43.943686 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.943605    2563 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 10 00:49:43.943686 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.943639    2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 10 00:49:43.943686 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.943656    2563 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 00:49:43.944299 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.944280    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-204.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 00:49:43.944348 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.944304    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-204.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 00:49:43.944348 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.944316    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-204.ec2.internal" event="NodeHasSufficientPID"
Sep 10 00:49:43.944782 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:43.944768    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hzcxr"
Sep 10 00:49:43.952800 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:43.952725    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-204.ec2.internal.18d3cf47b899091f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-204.ec2.internal,UID:ip-10-0-138-204.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-204.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-204.ec2.internal,},FirstTimestamp:2026-09-10 00:49:43.942883615 +0000 UTC m=+0.440183188,LastTimestamp:2026-09-10 00:49:43.942883615 +0000 UTC m=+0.440183188,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-204.ec2.internal,}"
Sep 10 00:49:43.955192 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:43.955129    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-204.ec2.internal.18d3cf47b8996e59  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-204.ec2.internal,UID:ip-10-0-138-204.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-138-204.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-138-204.ec2.internal,},FirstTimestamp:2026-09-10 00:49:43.942909529 +0000 UTC m=+0.440209101,LastTimestamp:2026-09-10 00:49:43.942909529 +0000 UTC m=+0.440209101,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-204.ec2.internal,}"
Sep 10 00:49:43.963719 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:43.963639    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-204.ec2.internal.18d3cf47b899a588  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-204.ec2.internal,UID:ip-10-0-138-204.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-138-204.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-138-204.ec2.internal,},FirstTimestamp:2026-09-10 00:49:43.942923656 +0000 UTC m=+0.440223226,LastTimestamp:2026-09-10 00:49:43.942923656 +0000 UTC m=+0.440223226,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-204.ec2.internal,}"
Sep 10 00:49:43.973662 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:43.973596    2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-138-204.ec2.internal.18d3cf47b899091f\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-204.ec2.internal.18d3cf47b899091f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-204.ec2.internal,UID:ip-10-0-138-204.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-204.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-204.ec2.internal,},FirstTimestamp:2026-09-10 00:49:43.942883615 +0000 UTC m=+0.440183188,LastTimestamp:2026-09-10 00:49:43.944292978 +0000 UTC m=+0.441592548,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-204.ec2.internal,}"
Sep 10 00:49:43.999846 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:43.981889    2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-138-204.ec2.internal.18d3cf47b8996e59\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-204.ec2.internal.18d3cf47b8996e59  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-204.ec2.internal,UID:ip-10-0-138-204.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-138-204.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-138-204.ec2.internal,},FirstTimestamp:2026-09-10 00:49:43.942909529 +0000 UTC m=+0.440209101,LastTimestamp:2026-09-10 00:49:43.94430853 +0000 UTC m=+0.441608100,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-204.ec2.internal,}"
Sep 10 00:49:43.999846 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:43.990113    2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-138-204.ec2.internal.18d3cf47b899a588\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-204.ec2.internal.18d3cf47b899a588  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-204.ec2.internal,UID:ip-10-0-138-204.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-138-204.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-138-204.ec2.internal,},FirstTimestamp:2026-09-10 00:49:43.942923656 +0000 UTC m=+0.440223226,LastTimestamp:2026-09-10 00:49:43.944321775 +0000 UTC m=+0.441621345,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-204.ec2.internal,}"
Sep 10 00:49:44.020277 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:44.020249    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-204.ec2.internal\" not found"
Sep 10 00:49:44.061534 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.061503    2563 policy_none.go:49] "None policy: Start"
Sep 10 00:49:44.061674 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.061552    2563 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 10 00:49:44.061674 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.061573    2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 10 00:49:44.063871 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.063854    2563 policy_none.go:47] "Start"
Sep 10 00:49:44.074377 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.074352    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 10 00:49:44.075659 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.075636    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 10 00:49:44.075755 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.075696    2563 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 10 00:49:44.075755 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.075728    2563 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 10 00:49:44.075849 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:44.075774    2563 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Sep 10 00:49:44.086678 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.086656    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 00:49:44.095600 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.095588    2563 manager.go:337] "Starting Device Plugin manager"
Sep 10 00:49:44.095677 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:44.095625    2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 10 00:49:44.095677 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.095635    2563 server.go:87] "Starting device plugin registration server"
Sep 10 00:49:44.096077 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.096063    2563 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 10 00:49:44.096146 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.096082    2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 10 00:49:44.096208 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.096192    2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 10 00:49:44.096332 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.096316    2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 10 00:49:44.096332 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.096329    2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 10 00:49:44.096910 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:44.096893    2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 10 00:49:44.096997 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:44.096939    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-204.ec2.internal\" not found"
Sep 10 00:49:44.141028 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:44.140996    2563 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-138-204.ec2.internal\" not found" node="ip-10-0-138-204.ec2.internal"
Sep 10 00:49:44.176153 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.176114    2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-204.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-204.ec2.internal"]
Sep 10 00:49:44.176229 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.176212    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 00:49:44.177273 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.177247    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-204.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 00:49:44.177387 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.177281    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-204.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 00:49:44.177387 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.177297    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-204.ec2.internal" event="NodeHasSufficientPID"
Sep 10 00:49:44.179820 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.179807    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 00:49:44.179930 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.179915    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-204.ec2.internal"
Sep 10 00:49:44.179970 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.179945    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 00:49:44.180645 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.180630    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-204.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 00:49:44.180719 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.180653    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-204.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 00:49:44.180719 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.180668    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-204.ec2.internal" event="NodeHasSufficientPID"
Sep 10 00:49:44.180719 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.180632    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-204.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 00:49:44.180719 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.180704    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-204.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 00:49:44.180719 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.180715    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-204.ec2.internal" event="NodeHasSufficientPID"
Sep 10 00:49:44.183081 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.183066    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-204.ec2.internal"
Sep 10 00:49:44.183162 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.183096    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 00:49:44.183810 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.183793    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-204.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 00:49:44.183894 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.183826    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-204.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 00:49:44.183894 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.183840    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-204.ec2.internal" event="NodeHasSufficientPID"
Sep 10 00:49:44.197039 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.197020    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 00:49:44.199972 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.199955    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-204.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 00:49:44.200077 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.200003    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-204.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 00:49:44.200077 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.200020    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-204.ec2.internal" event="NodeHasSufficientPID"
Sep 10 00:49:44.200077 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.200046    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-204.ec2.internal"
Sep 10 00:49:44.203634 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:44.203619    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-204.ec2.internal\" not found" node="ip-10-0-138-204.ec2.internal"
Sep 10 00:49:44.208272 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:44.208256    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-204.ec2.internal\" not found" node="ip-10-0-138-204.ec2.internal"
Sep 10 00:49:44.208357 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.208344    2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-204.ec2.internal"
Sep 10 00:49:44.208395 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:44.208363    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-204.ec2.internal\": node \"ip-10-0-138-204.ec2.internal\" not found"
Sep 10 00:49:44.222787 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.222765    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/11a33808a44d7e8ec5f92874d5f9136a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-204.ec2.internal\" (UID: \"11a33808a44d7e8ec5f92874d5f9136a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-204.ec2.internal"
Sep 10 00:49:44.222869 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.222793    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11a33808a44d7e8ec5f92874d5f9136a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-204.ec2.internal\" (UID: \"11a33808a44d7e8ec5f92874d5f9136a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-204.ec2.internal"
Sep 10 00:49:44.222869 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.222810    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4ce798d5035df86fd4210aa6d5fc499e-config\") pod \"kube-apiserver-proxy-ip-10-0-138-204.ec2.internal\" (UID: \"4ce798d5035df86fd4210aa6d5fc499e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-204.ec2.internal"
Sep 10 00:49:44.226057 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:44.226041    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-204.ec2.internal\" not found"
Sep 10 00:49:44.323421 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.323381    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4ce798d5035df86fd4210aa6d5fc499e-config\") pod \"kube-apiserver-proxy-ip-10-0-138-204.ec2.internal\" (UID: \"4ce798d5035df86fd4210aa6d5fc499e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-204.ec2.internal"
Sep 10 00:49:44.323599 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.323431    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/11a33808a44d7e8ec5f92874d5f9136a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-204.ec2.internal\" (UID: \"11a33808a44d7e8ec5f92874d5f9136a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-204.ec2.internal"
Sep 10 00:49:44.323599 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.323453    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11a33808a44d7e8ec5f92874d5f9136a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-204.ec2.internal\" (UID: \"11a33808a44d7e8ec5f92874d5f9136a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-204.ec2.internal"
Sep 10 00:49:44.323599 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.323475    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4ce798d5035df86fd4210aa6d5fc499e-config\") pod \"kube-apiserver-proxy-ip-10-0-138-204.ec2.internal\" (UID: \"4ce798d5035df86fd4210aa6d5fc499e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-204.ec2.internal"
Sep 10 00:49:44.323599 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.323527    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/11a33808a44d7e8ec5f92874d5f9136a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-204.ec2.internal\" (UID: \"11a33808a44d7e8ec5f92874d5f9136a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-204.ec2.internal"
Sep 10 00:49:44.323599 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.323544    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11a33808a44d7e8ec5f92874d5f9136a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-204.ec2.internal\" (UID: \"11a33808a44d7e8ec5f92874d5f9136a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-204.ec2.internal"
Sep 10 00:49:44.326469 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:44.326450    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-204.ec2.internal\" not found"
Sep 10 00:49:44.427412 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:44.427338    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-204.ec2.internal\" not found"
Sep 10 00:49:44.506574 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.506537    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-204.ec2.internal"
Sep 10 00:49:44.512693 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.512667    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-204.ec2.internal"
Sep 10 00:49:44.528345 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:44.528319    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-204.ec2.internal\" not found"
Sep 10 00:49:44.628620 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:44.628580    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-204.ec2.internal\" not found"
Sep 10 00:49:44.729649 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:44.729555    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-204.ec2.internal\" not found"
Sep 10 00:49:44.830613 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:44.830579    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-204.ec2.internal\" not found"
Sep 10 00:49:44.853401 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.853371    2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 10 00:49:44.853543 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:44.853526    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 00:49:44.931140 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:44.931109    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-204.ec2.internal\" not found"
Sep 10 00:49:45.031258 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:45.031221    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-204.ec2.internal\" not found"
Sep 10 00:49:45.066451 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.066401    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 00:44:44 +0000 UTC" deadline="2028-04-11 14:44:29.155241535 +0000 UTC"
Sep 10 00:49:45.066451 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.066449    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13909h54m44.088798267s"
Sep 10 00:49:45.090416 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.090386    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 00:49:45.119896 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.119852    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-204.ec2.internal"
Sep 10 00:49:45.133692 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.133667    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 00:49:45.135836 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.135821    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-204.ec2.internal"
Sep 10 00:49:45.143824 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.143803    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 00:49:45.145044 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.145025    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 00:49:45.146180 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:45.146156    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ce798d5035df86fd4210aa6d5fc499e.slice/crio-b378934ed39a6cafa581a3a6aeb8e04f10cc6e73fa69d9b9d696a7f92c1543d1 WatchSource:0}: Error finding container b378934ed39a6cafa581a3a6aeb8e04f10cc6e73fa69d9b9d696a7f92c1543d1: Status 404 returned error can't find the container with id b378934ed39a6cafa581a3a6aeb8e04f10cc6e73fa69d9b9d696a7f92c1543d1
Sep 10 00:49:45.151508 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.151492    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 00:49:45.228147 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.228108    2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 00:49:45.407499 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:45.407470    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11a33808a44d7e8ec5f92874d5f9136a.slice/crio-629916ec8f8b47e3411169e8f1f1af1411a82875adf35ae6279325eb7cf9fbef WatchSource:0}: Error finding container 629916ec8f8b47e3411169e8f1f1af1411a82875adf35ae6279325eb7cf9fbef: Status 404 returned error can't find the container with id 629916ec8f8b47e3411169e8f1f1af1411a82875adf35ae6279325eb7cf9fbef
Sep 10 00:49:45.898703 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.898619    2563 apiserver.go:52] "Watching apiserver"
Sep 10 00:49:45.906479 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.906445    2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 10 00:49:45.907831 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.907801    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-8nwz8","kube-system/global-pull-secret-syncer-l4w6p","kube-system/kube-apiserver-proxy-ip-10-0-138-204.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f27tb","openshift-cluster-node-tuning-operator/tuned-8qrc5","openshift-image-registry/node-ca-tr2kj","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-204.ec2.internal","openshift-multus/multus-9zt49","openshift-multus/multus-additional-cni-plugins-6nzvr","kube-system/konnectivity-agent-qhbzc","openshift-dns/node-resolver-p5zpt","openshift-multus/network-metrics-daemon-8ltcj","openshift-network-diagnostics/network-check-target-cvjrx","openshift-network-operator/iptables-alerter-cncq7"]
Sep 10 00:49:45.910629 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.910605    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:45.913073 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.913036    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l4w6p"
Sep 10 00:49:45.913189 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:45.913134    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l4w6p" podUID="00cec203-bdcf-4e75-acfa-29e1b784840e"
Sep 10 00:49:45.913636 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.913465    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 10 00:49:45.913636 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.913500    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 10 00:49:45.913636 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.913547    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 10 00:49:45.914072 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.913913    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 10 00:49:45.914072 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.913927    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-l7rtj\""
Sep 10 00:49:45.914072 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.913938    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 10 00:49:45.914328 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.914308    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 10 00:49:45.915551 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.915474    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f27tb"
Sep 10 00:49:45.918191 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.917872    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 10 00:49:45.918191 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.917971    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:45.918324 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.918244    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-g7x29\""
Sep 10 00:49:45.918521 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.918503    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 10 00:49:45.918606 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.918507    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 10 00:49:45.920688 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.920668    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 10 00:49:45.920805 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.920787    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tr2kj"
Sep 10 00:49:45.921003 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.920984    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-q7qsm\""
Sep 10 00:49:45.921119 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.921066    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 10 00:49:45.921183 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.920990    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 10 00:49:45.923102 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.923061    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9zt49"
Sep 10 00:49:45.923547 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.923526    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 10 00:49:45.923998 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.923854    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 10 00:49:45.923998 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.923883    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 10 00:49:45.923998 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.923904    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-fv4tk\""
Sep 10 00:49:45.925903 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.925613    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6nzvr"
Sep 10 00:49:45.925987 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.925941    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 10 00:49:45.926080 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.926047    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 10 00:49:45.926187 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.926167    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 10 00:49:45.926394 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.926340    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-4755t\""
Sep 10 00:49:45.926394 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.926388    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 10 00:49:45.928079 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.928061    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qhbzc"
Sep 10 00:49:45.928368 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.928345    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 10 00:49:45.928943 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.928874    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zdqf5\""
Sep 10 00:49:45.928943 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.928903    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 10 00:49:45.931140 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.931119    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-p5zpt"
Sep 10 00:49:45.931242 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.931146    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 10 00:49:45.931473 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.931450    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-rwhjs\""
Sep 10 00:49:45.931688 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.931667    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 10 00:49:45.932396 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.932373    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-host-slash\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:45.932497 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.932415    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-host-run-ovn-kubernetes\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:45.932497 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.932451    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/283dd2c2-21ef-431b-893d-b9969c7471a7-etc-systemd\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:45.932497 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.932481    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/283dd2c2-21ef-431b-893d-b9969c7471a7-run\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:45.932665 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.932509    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/283dd2c2-21ef-431b-893d-b9969c7471a7-tmp\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:45.932665 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.932541    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/86435252-3772-46f1-a9a3-c8a229064702-registration-dir\") pod \"aws-ebs-csi-driver-node-f27tb\" (UID: \"86435252-3772-46f1-a9a3-c8a229064702\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f27tb"
Sep 10 00:49:45.932665 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.932580    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8686a32b-9d47-4866-aae0-89f0b6575ff3-host\") pod \"node-ca-tr2kj\" (UID: \"8686a32b-9d47-4866-aae0-89f0b6575ff3\") " pod="openshift-image-registry/node-ca-tr2kj"
Sep 10 00:49:45.932665 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.932610    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpq6d\" (UniqueName: \"kubernetes.io/projected/7a312259-8fb8-4bb4-9304-0b5aac8240f2-kube-api-access-dpq6d\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:45.932665 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.932658    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/00cec203-bdcf-4e75-acfa-29e1b784840e-original-pull-secret\") pod \"global-pull-secret-syncer-l4w6p\" (UID: \"00cec203-bdcf-4e75-acfa-29e1b784840e\") " pod="kube-system/global-pull-secret-syncer-l4w6p"
Sep 10 00:49:45.932992 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.932690    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/283dd2c2-21ef-431b-893d-b9969c7471a7-etc-modprobe-d\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:45.932992 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.932719    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/283dd2c2-21ef-431b-893d-b9969c7471a7-etc-tuned\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:45.932992 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.932800    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4d429\" (UniqueName: \"kubernetes.io/projected/8686a32b-9d47-4866-aae0-89f0b6575ff3-kube-api-access-4d429\") pod \"node-ca-tr2kj\" (UID: \"8686a32b-9d47-4866-aae0-89f0b6575ff3\") " pod="openshift-image-registry/node-ca-tr2kj"
Sep 10 00:49:45.932992 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.932833    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-host-var-lib-kubelet\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:45.932992 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.932891    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/981f2019-8639-47c8-b1d3-f2dbb02acd15-env-overrides\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:45.932992 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.932896    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 00:49:45.932992 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.932922    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/86435252-3772-46f1-a9a3-c8a229064702-kubelet-dir\") pod \"aws-ebs-csi-driver-node-f27tb\" (UID: \"86435252-3772-46f1-a9a3-c8a229064702\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f27tb"
Sep 10 00:49:45.932992 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.932955    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/283dd2c2-21ef-431b-893d-b9969c7471a7-etc-sysctl-d\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:45.933413 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.933016    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/00cec203-bdcf-4e75-acfa-29e1b784840e-dbus\") pod \"global-pull-secret-syncer-l4w6p\" (UID: \"00cec203-bdcf-4e75-acfa-29e1b784840e\") " pod="kube-system/global-pull-secret-syncer-l4w6p"
Sep 10 00:49:45.933413 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.933096    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s52dd\" (UniqueName: \"kubernetes.io/projected/86435252-3772-46f1-a9a3-c8a229064702-kube-api-access-s52dd\") pod \"aws-ebs-csi-driver-node-f27tb\" (UID: \"86435252-3772-46f1-a9a3-c8a229064702\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f27tb"
Sep 10 00:49:45.933413 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.933136    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/283dd2c2-21ef-431b-893d-b9969c7471a7-sys\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:45.933413 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.933191    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-cnibin\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:45.933413 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.933226    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-multus-conf-dir\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:45.933413 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.933256    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/86435252-3772-46f1-a9a3-c8a229064702-etc-selinux\") pod \"aws-ebs-csi-driver-node-f27tb\" (UID: \"86435252-3772-46f1-a9a3-c8a229064702\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f27tb"
Sep 10 00:49:45.933413 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.933289    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-host-var-lib-cni-multus\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:45.933413 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.933339    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-host-run-netns\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:45.933413 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.933367    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:45.933413 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.933397    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfxj8\" (UniqueName: \"kubernetes.io/projected/981f2019-8639-47c8-b1d3-f2dbb02acd15-kube-api-access-jfxj8\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:45.933908 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.933425    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58cp4\" (UniqueName: \"kubernetes.io/projected/283dd2c2-21ef-431b-893d-b9969c7471a7-kube-api-access-58cp4\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:45.933908 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.933455    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7a312259-8fb8-4bb4-9304-0b5aac8240f2-cni-binary-copy\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:45.933908 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.933485    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-host-var-lib-cni-bin\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:45.933908 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.933528    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-host-run-multus-certs\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:45.933908 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.933597    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-run-systemd\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:45.933908 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.933643    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/981f2019-8639-47c8-b1d3-f2dbb02acd15-ovnkube-config\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:45.933908 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.933674    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/981f2019-8639-47c8-b1d3-f2dbb02acd15-ovn-node-metrics-cert\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:45.933908 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.933716    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/981f2019-8639-47c8-b1d3-f2dbb02acd15-ovnkube-script-lib\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:45.934358 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.934123    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 10 00:49:45.934765 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.934745    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 10 00:49:45.934938 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.934911    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ft7rp\""
Sep 10 00:49:45.935517 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.935494    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/86435252-3772-46f1-a9a3-c8a229064702-sys-fs\") pod \"aws-ebs-csi-driver-node-f27tb\" (UID: \"86435252-3772-46f1-a9a3-c8a229064702\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f27tb"
Sep 10 00:49:45.935608 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.935537    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/283dd2c2-21ef-431b-893d-b9969c7471a7-etc-sysconfig\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:45.935608 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.935566    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/283dd2c2-21ef-431b-893d-b9969c7471a7-host\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:45.935724 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.935619    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/86435252-3772-46f1-a9a3-c8a229064702-socket-dir\") pod \"aws-ebs-csi-driver-node-f27tb\" (UID: \"86435252-3772-46f1-a9a3-c8a229064702\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f27tb"
Sep 10 00:49:45.935724 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.935647    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/283dd2c2-21ef-431b-893d-b9969c7471a7-lib-modules\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:45.935724 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.935679    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8686a32b-9d47-4866-aae0-89f0b6575ff3-serviceca\") pod \"node-ca-tr2kj\" (UID: \"8686a32b-9d47-4866-aae0-89f0b6575ff3\") " pod="openshift-image-registry/node-ca-tr2kj"
Sep 10 00:49:45.935875 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.935746    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7a312259-8fb8-4bb4-9304-0b5aac8240f2-multus-daemon-config\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:45.935875 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.935785    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-node-log\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:45.935875 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.935814    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-host-run-netns\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:45.935875 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.935848    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-run-openvswitch\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:45.936148 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.935881    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-run-ovn\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:45.936148 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.935950    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-etc-kubernetes\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:45.936148 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.936100    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-var-lib-openvswitch\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:45.936148 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.936140    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-host-cni-bin\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:45.936360 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.936188    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/283dd2c2-21ef-431b-893d-b9969c7471a7-etc-sysctl-conf\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:45.936360 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.936231    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-multus-socket-dir-parent\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:45.936360 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.936282    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-hostroot\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:45.936360 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.936314    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/283dd2c2-21ef-431b-893d-b9969c7471a7-etc-kubernetes\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:45.936360 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.936341    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/283dd2c2-21ef-431b-893d-b9969c7471a7-var-lib-kubelet\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:45.936605 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.936367    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-multus-cni-dir\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:45.936605 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.936397    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-systemd-units\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:45.937090 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.936836    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-etc-openvswitch\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:45.937090 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.936882    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-log-socket\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:45.937090 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.936942    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/00cec203-bdcf-4e75-acfa-29e1b784840e-kubelet-config\") pod \"global-pull-secret-syncer-l4w6p\" (UID: \"00cec203-bdcf-4e75-acfa-29e1b784840e\") " pod="kube-system/global-pull-secret-syncer-l4w6p"
Sep 10 00:49:45.937090 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.936978    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-system-cni-dir\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:45.937090 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.937003    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-host-run-k8s-cni-cncf-io\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:45.937090 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.937042    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-host-kubelet\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:45.937090 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.937087    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-host-cni-netd\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:45.937540 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.937104    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/86435252-3772-46f1-a9a3-c8a229064702-device-dir\") pod \"aws-ebs-csi-driver-node-f27tb\" (UID: \"86435252-3772-46f1-a9a3-c8a229064702\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f27tb"
Sep 10 00:49:45.937540 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.937373    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltcj"
Sep 10 00:49:45.937540 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:45.937462    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8ltcj" podUID="fc958bde-e5aa-468a-ae10-0c49f9438168"
Sep 10 00:49:45.940171 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.940150    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cvjrx"
Sep 10 00:49:45.940269 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:45.940213    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cvjrx" podUID="50d35dde-26ab-4a1a-8f6a-2d663df04bc5"
Sep 10 00:49:45.942642 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.942620    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cncq7"
Sep 10 00:49:45.945339 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.945322    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 10 00:49:45.945339 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.945328    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 10 00:49:45.945535 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.945478    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-88wmw\""
Sep 10 00:49:45.945632 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.945613    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 10 00:49:45.958562 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.958535    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ffbft"
Sep 10 00:49:45.967559 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.967537    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ffbft"
Sep 10 00:49:45.987338 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:45.987312    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 00:49:46.022167 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.022142    2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 10 00:49:46.038303 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.038261    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/00cec203-bdcf-4e75-acfa-29e1b784840e-dbus\") pod \"global-pull-secret-syncer-l4w6p\" (UID: \"00cec203-bdcf-4e75-acfa-29e1b784840e\") " pod="kube-system/global-pull-secret-syncer-l4w6p"
Sep 10 00:49:46.038476 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.038308    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s52dd\" (UniqueName: \"kubernetes.io/projected/86435252-3772-46f1-a9a3-c8a229064702-kube-api-access-s52dd\") pod \"aws-ebs-csi-driver-node-f27tb\" (UID: \"86435252-3772-46f1-a9a3-c8a229064702\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f27tb"
Sep 10 00:49:46.038476 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.038337    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/283dd2c2-21ef-431b-893d-b9969c7471a7-sys\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:46.038476 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.038369    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-cnibin\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:46.038476 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.038394    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-multus-conf-dir\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:46.038476 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.038393    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/00cec203-bdcf-4e75-acfa-29e1b784840e-dbus\") pod \"global-pull-secret-syncer-l4w6p\" (UID: \"00cec203-bdcf-4e75-acfa-29e1b784840e\") " pod="kube-system/global-pull-secret-syncer-l4w6p"
Sep 10 00:49:46.038476 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.038421    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/86435252-3772-46f1-a9a3-c8a229064702-etc-selinux\") pod \"aws-ebs-csi-driver-node-f27tb\" (UID: \"86435252-3772-46f1-a9a3-c8a229064702\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f27tb"
Sep 10 00:49:46.038476 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.038446    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-host-var-lib-cni-multus\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:46.038842 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.038477    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/283dd2c2-21ef-431b-893d-b9969c7471a7-sys\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:46.038842 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.038477    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-multus-conf-dir\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:46.038842 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.038477    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqqdw\" (UniqueName: \"kubernetes.io/projected/27351a7f-151c-4d9a-bddd-4ebaa3a42ade-kube-api-access-rqqdw\") pod \"node-resolver-p5zpt\" (UID: \"27351a7f-151c-4d9a-bddd-4ebaa3a42ade\") " pod="openshift-dns/node-resolver-p5zpt"
Sep 10 00:49:46.038842 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.038540    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-cnibin\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:46.038842 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.038542    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/86435252-3772-46f1-a9a3-c8a229064702-etc-selinux\") pod \"aws-ebs-csi-driver-node-f27tb\" (UID: \"86435252-3772-46f1-a9a3-c8a229064702\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f27tb"
Sep 10 00:49:46.038842 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.038579    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-host-var-lib-cni-multus\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:46.038842 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.038643    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-host-run-netns\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.038842 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.038681    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.038842 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.038711    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfxj8\" (UniqueName: \"kubernetes.io/projected/981f2019-8639-47c8-b1d3-f2dbb02acd15-kube-api-access-jfxj8\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.038842 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.038762    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58cp4\" (UniqueName: \"kubernetes.io/projected/283dd2c2-21ef-431b-893d-b9969c7471a7-kube-api-access-58cp4\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:46.038842 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.038789    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7a312259-8fb8-4bb4-9304-0b5aac8240f2-cni-binary-copy\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:46.038842 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.038836    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-host-var-lib-cni-bin\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:46.039458 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.038870    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.039458 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.038930    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-host-run-netns\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.039458 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.038998    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-host-run-multus-certs\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:46.039458 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.039030    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-run-systemd\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.039458 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.039084    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/981f2019-8639-47c8-b1d3-f2dbb02acd15-ovnkube-config\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.039458 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.039103    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-host-var-lib-cni-bin\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:46.039458 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.039120    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/981f2019-8639-47c8-b1d3-f2dbb02acd15-ovn-node-metrics-cert\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.039458 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.039145    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-host-run-multus-certs\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:46.039458 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.039185    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/981f2019-8639-47c8-b1d3-f2dbb02acd15-ovnkube-script-lib\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.039458 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.039211    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/86435252-3772-46f1-a9a3-c8a229064702-sys-fs\") pod \"aws-ebs-csi-driver-node-f27tb\" (UID: \"86435252-3772-46f1-a9a3-c8a229064702\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f27tb"
Sep 10 00:49:46.039458 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.039239    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/283dd2c2-21ef-431b-893d-b9969c7471a7-etc-sysconfig\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:46.039458 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.039264    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/283dd2c2-21ef-431b-893d-b9969c7471a7-host\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:46.039458 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.039308    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/283dd2c2-21ef-431b-893d-b9969c7471a7-host\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:46.039458 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.039356    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/86435252-3772-46f1-a9a3-c8a229064702-sys-fs\") pod \"aws-ebs-csi-driver-node-f27tb\" (UID: \"86435252-3772-46f1-a9a3-c8a229064702\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f27tb"
Sep 10 00:49:46.039458 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.039388    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-run-systemd\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.039458 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.039450    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/283dd2c2-21ef-431b-893d-b9969c7471a7-etc-sysconfig\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:46.040198 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.039533    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/afe1ccfd-10c6-48e7-83aa-1d1384f532e8-host-slash\") pod \"iptables-alerter-cncq7\" (UID: \"afe1ccfd-10c6-48e7-83aa-1d1384f532e8\") " pod="openshift-network-operator/iptables-alerter-cncq7"
Sep 10 00:49:46.040198 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.039574    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/86435252-3772-46f1-a9a3-c8a229064702-socket-dir\") pod \"aws-ebs-csi-driver-node-f27tb\" (UID: \"86435252-3772-46f1-a9a3-c8a229064702\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f27tb"
Sep 10 00:49:46.040198 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.039608    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/283dd2c2-21ef-431b-893d-b9969c7471a7-lib-modules\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:46.040198 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.039647    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7a312259-8fb8-4bb4-9304-0b5aac8240f2-cni-binary-copy\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:46.040198 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.039733    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/86435252-3772-46f1-a9a3-c8a229064702-socket-dir\") pod \"aws-ebs-csi-driver-node-f27tb\" (UID: \"86435252-3772-46f1-a9a3-c8a229064702\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f27tb"
Sep 10 00:49:46.040198 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.039761    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8686a32b-9d47-4866-aae0-89f0b6575ff3-serviceca\") pod \"node-ca-tr2kj\" (UID: \"8686a32b-9d47-4866-aae0-89f0b6575ff3\") " pod="openshift-image-registry/node-ca-tr2kj"
Sep 10 00:49:46.040198 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.039799    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/283dd2c2-21ef-431b-893d-b9969c7471a7-lib-modules\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:46.040198 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.039806    2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 10 00:49:46.040198 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.039893    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/981f2019-8639-47c8-b1d3-f2dbb02acd15-ovnkube-config\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.040198 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.039804    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7a312259-8fb8-4bb4-9304-0b5aac8240f2-multus-daemon-config\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:46.040198 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.039946    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc958bde-e5aa-468a-ae10-0c49f9438168-metrics-certs\") pod \"network-metrics-daemon-8ltcj\" (UID: \"fc958bde-e5aa-468a-ae10-0c49f9438168\") " pod="openshift-multus/network-metrics-daemon-8ltcj"
Sep 10 00:49:46.040198 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.039974    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9f66caff-9501-4b7e-8986-9161a3ae70e3-system-cni-dir\") pod \"multus-additional-cni-plugins-6nzvr\" (UID: \"9f66caff-9501-4b7e-8986-9161a3ae70e3\") " pod="openshift-multus/multus-additional-cni-plugins-6nzvr"
Sep 10 00:49:46.040198 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040021    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9f66caff-9501-4b7e-8986-9161a3ae70e3-cnibin\") pod \"multus-additional-cni-plugins-6nzvr\" (UID: \"9f66caff-9501-4b7e-8986-9161a3ae70e3\") " pod="openshift-multus/multus-additional-cni-plugins-6nzvr"
Sep 10 00:49:46.040198 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040045    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/caff9962-15c8-4c05-ad49-a748c94a3b85-konnectivity-ca\") pod \"konnectivity-agent-qhbzc\" (UID: \"caff9962-15c8-4c05-ad49-a748c94a3b85\") " pod="kube-system/konnectivity-agent-qhbzc"
Sep 10 00:49:46.040198 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040085    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/981f2019-8639-47c8-b1d3-f2dbb02acd15-ovnkube-script-lib\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.040198 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040073    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-node-log\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.040198 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040131    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-node-log\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.040198 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040176    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-host-run-netns\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:46.041192 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040220    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbcfv\" (UniqueName: \"kubernetes.io/projected/afe1ccfd-10c6-48e7-83aa-1d1384f532e8-kube-api-access-vbcfv\") pod \"iptables-alerter-cncq7\" (UID: \"afe1ccfd-10c6-48e7-83aa-1d1384f532e8\") " pod="openshift-network-operator/iptables-alerter-cncq7"
Sep 10 00:49:46.041192 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040234    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8686a32b-9d47-4866-aae0-89f0b6575ff3-serviceca\") pod \"node-ca-tr2kj\" (UID: \"8686a32b-9d47-4866-aae0-89f0b6575ff3\") " pod="openshift-image-registry/node-ca-tr2kj"
Sep 10 00:49:46.041192 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040249    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9f66caff-9501-4b7e-8986-9161a3ae70e3-cni-binary-copy\") pod \"multus-additional-cni-plugins-6nzvr\" (UID: \"9f66caff-9501-4b7e-8986-9161a3ae70e3\") " pod="openshift-multus/multus-additional-cni-plugins-6nzvr"
Sep 10 00:49:46.041192 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040261    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-host-run-netns\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:46.041192 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040287    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9f66caff-9501-4b7e-8986-9161a3ae70e3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6nzvr\" (UID: \"9f66caff-9501-4b7e-8986-9161a3ae70e3\") " pod="openshift-multus/multus-additional-cni-plugins-6nzvr"
Sep 10 00:49:46.041192 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040317    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/caff9962-15c8-4c05-ad49-a748c94a3b85-agent-certs\") pod \"konnectivity-agent-qhbzc\" (UID: \"caff9962-15c8-4c05-ad49-a748c94a3b85\") " pod="kube-system/konnectivity-agent-qhbzc"
Sep 10 00:49:46.041192 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040317    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7a312259-8fb8-4bb4-9304-0b5aac8240f2-multus-daemon-config\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:46.041192 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040346    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-run-openvswitch\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.041192 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040375    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-run-ovn\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.041192 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040418    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-run-openvswitch\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.041192 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040421    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-run-ovn\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.041192 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040458    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-etc-kubernetes\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:46.041192 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040480    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-etc-kubernetes\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:46.041192 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040488    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8csw6\" (UniqueName: \"kubernetes.io/projected/fc958bde-e5aa-468a-ae10-0c49f9438168-kube-api-access-8csw6\") pod \"network-metrics-daemon-8ltcj\" (UID: \"fc958bde-e5aa-468a-ae10-0c49f9438168\") " pod="openshift-multus/network-metrics-daemon-8ltcj"
Sep 10 00:49:46.041192 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040519    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-var-lib-openvswitch\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.041192 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040544    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-host-cni-bin\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.041192 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040567    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/283dd2c2-21ef-431b-893d-b9969c7471a7-etc-sysctl-conf\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:46.042020 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040593    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-multus-socket-dir-parent\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:46.042020 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040596    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-var-lib-openvswitch\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.042020 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040618    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-hostroot\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:46.042020 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040644    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/283dd2c2-21ef-431b-893d-b9969c7471a7-etc-kubernetes\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:46.042020 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040667    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/283dd2c2-21ef-431b-893d-b9969c7471a7-var-lib-kubelet\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:46.042020 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040710    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-multus-cni-dir\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:46.042020 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040735    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-systemd-units\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.042020 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040769    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/283dd2c2-21ef-431b-893d-b9969c7471a7-etc-sysctl-conf\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:46.042020 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040780    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-etc-openvswitch\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.042020 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040805    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-log-socket\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.042020 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040813    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-host-cni-bin\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.042020 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040835    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27351a7f-151c-4d9a-bddd-4ebaa3a42ade-tmp-dir\") pod \"node-resolver-p5zpt\" (UID: \"27351a7f-151c-4d9a-bddd-4ebaa3a42ade\") " pod="openshift-dns/node-resolver-p5zpt"
Sep 10 00:49:46.042020 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040864    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/00cec203-bdcf-4e75-acfa-29e1b784840e-kubelet-config\") pod \"global-pull-secret-syncer-l4w6p\" (UID: \"00cec203-bdcf-4e75-acfa-29e1b784840e\") " pod="kube-system/global-pull-secret-syncer-l4w6p"
Sep 10 00:49:46.042020 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040868    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/283dd2c2-21ef-431b-893d-b9969c7471a7-var-lib-kubelet\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:46.042020 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040891    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-system-cni-dir\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:46.042020 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040916    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-host-run-k8s-cni-cncf-io\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:46.042020 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040923    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-multus-socket-dir-parent\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:46.042020 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040943    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-host-kubelet\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.043014 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040966    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-hostroot\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:46.043014 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040972    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-host-cni-netd\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.043014 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.040999    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/86435252-3772-46f1-a9a3-c8a229064702-device-dir\") pod \"aws-ebs-csi-driver-node-f27tb\" (UID: \"86435252-3772-46f1-a9a3-c8a229064702\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f27tb"
Sep 10 00:49:46.043014 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.041023    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-host-slash\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.043014 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.041056    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/283dd2c2-21ef-431b-893d-b9969c7471a7-etc-kubernetes\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:46.043014 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.041064    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-host-run-ovn-kubernetes\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.043014 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.041112    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/283dd2c2-21ef-431b-893d-b9969c7471a7-etc-systemd\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:46.043014 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.041117    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-system-cni-dir\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:46.043014 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.041147    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/283dd2c2-21ef-431b-893d-b9969c7471a7-run\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:46.043014 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.041176    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/283dd2c2-21ef-431b-893d-b9969c7471a7-etc-systemd\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:46.043014 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.041174    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/283dd2c2-21ef-431b-893d-b9969c7471a7-tmp\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:46.043014 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.041216    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/afe1ccfd-10c6-48e7-83aa-1d1384f532e8-iptables-alerter-script\") pod \"iptables-alerter-cncq7\" (UID: \"afe1ccfd-10c6-48e7-83aa-1d1384f532e8\") " pod="openshift-network-operator/iptables-alerter-cncq7"
Sep 10 00:49:46.043014 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.041230    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-host-run-k8s-cni-cncf-io\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:46.043014 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.041262    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9f66caff-9501-4b7e-8986-9161a3ae70e3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6nzvr\" (UID: \"9f66caff-9501-4b7e-8986-9161a3ae70e3\") " pod="openshift-multus/multus-additional-cni-plugins-6nzvr"
Sep 10 00:49:46.043014 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.041287    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-host-kubelet\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.043014 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.041292    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkfsl\" (UniqueName: \"kubernetes.io/projected/9f66caff-9501-4b7e-8986-9161a3ae70e3-kube-api-access-vkfsl\") pod \"multus-additional-cni-plugins-6nzvr\" (UID: \"9f66caff-9501-4b7e-8986-9161a3ae70e3\") " pod="openshift-multus/multus-additional-cni-plugins-6nzvr"
Sep 10 00:49:46.043014 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.041331    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-host-cni-netd\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.043859 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.041360    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/86435252-3772-46f1-a9a3-c8a229064702-registration-dir\") pod \"aws-ebs-csi-driver-node-f27tb\" (UID: \"86435252-3772-46f1-a9a3-c8a229064702\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f27tb"
Sep 10 00:49:46.043859 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.041401    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8686a32b-9d47-4866-aae0-89f0b6575ff3-host\") pod \"node-ca-tr2kj\" (UID: \"8686a32b-9d47-4866-aae0-89f0b6575ff3\") " pod="openshift-image-registry/node-ca-tr2kj"
Sep 10 00:49:46.043859 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.041419    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-etc-openvswitch\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.043859 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.041436    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpq6d\" (UniqueName: \"kubernetes.io/projected/7a312259-8fb8-4bb4-9304-0b5aac8240f2-kube-api-access-dpq6d\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:46.043859 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.041464    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9f66caff-9501-4b7e-8986-9161a3ae70e3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6nzvr\" (UID: \"9f66caff-9501-4b7e-8986-9161a3ae70e3\") " pod="openshift-multus/multus-additional-cni-plugins-6nzvr"
Sep 10 00:49:46.043859 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.041466    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-log-socket\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.043859 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.041494    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/00cec203-bdcf-4e75-acfa-29e1b784840e-original-pull-secret\") pod \"global-pull-secret-syncer-l4w6p\" (UID: \"00cec203-bdcf-4e75-acfa-29e1b784840e\") " pod="kube-system/global-pull-secret-syncer-l4w6p"
Sep 10 00:49:46.043859 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.041528    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/283dd2c2-21ef-431b-893d-b9969c7471a7-etc-modprobe-d\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:46.043859 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.041537    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/00cec203-bdcf-4e75-acfa-29e1b784840e-kubelet-config\") pod \"global-pull-secret-syncer-l4w6p\" (UID: \"00cec203-bdcf-4e75-acfa-29e1b784840e\") " pod="kube-system/global-pull-secret-syncer-l4w6p"
Sep 10 00:49:46.043859 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.041554    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/283dd2c2-21ef-431b-893d-b9969c7471a7-etc-tuned\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:46.043859 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.041581    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4d429\" (UniqueName: \"kubernetes.io/projected/8686a32b-9d47-4866-aae0-89f0b6575ff3-kube-api-access-4d429\") pod \"node-ca-tr2kj\" (UID: \"8686a32b-9d47-4866-aae0-89f0b6575ff3\") " pod="openshift-image-registry/node-ca-tr2kj"
Sep 10 00:49:46.043859 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.041583    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/283dd2c2-21ef-431b-893d-b9969c7471a7-run\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:46.043859 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.041607    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-host-var-lib-kubelet\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:46.043859 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.041634    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/27351a7f-151c-4d9a-bddd-4ebaa3a42ade-hosts-file\") pod \"node-resolver-p5zpt\" (UID: \"27351a7f-151c-4d9a-bddd-4ebaa3a42ade\") " pod="openshift-dns/node-resolver-p5zpt"
Sep 10 00:49:46.043859 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.041661    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/981f2019-8639-47c8-b1d3-f2dbb02acd15-env-overrides\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.043859 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.041685    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/86435252-3772-46f1-a9a3-c8a229064702-kubelet-dir\") pod \"aws-ebs-csi-driver-node-f27tb\" (UID: \"86435252-3772-46f1-a9a3-c8a229064702\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f27tb"
Sep 10 00:49:46.043859 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.041709    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/283dd2c2-21ef-431b-893d-b9969c7471a7-etc-sysctl-d\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:46.044581 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.041731    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kctp\" (UniqueName: \"kubernetes.io/projected/50d35dde-26ab-4a1a-8f6a-2d663df04bc5-kube-api-access-2kctp\") pod \"network-check-target-cvjrx\" (UID: \"50d35dde-26ab-4a1a-8f6a-2d663df04bc5\") " pod="openshift-network-diagnostics/network-check-target-cvjrx"
Sep 10 00:49:46.044581 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.041825    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/86435252-3772-46f1-a9a3-c8a229064702-device-dir\") pod \"aws-ebs-csi-driver-node-f27tb\" (UID: \"86435252-3772-46f1-a9a3-c8a229064702\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f27tb"
Sep 10 00:49:46.044581 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.041861    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-host-slash\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.044581 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.041896    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-host-run-ovn-kubernetes\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.044581 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.041965    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/86435252-3772-46f1-a9a3-c8a229064702-registration-dir\") pod \"aws-ebs-csi-driver-node-f27tb\" (UID: \"86435252-3772-46f1-a9a3-c8a229064702\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f27tb"
Sep 10 00:49:46.044581 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.041372    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-multus-cni-dir\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:46.044581 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.042025    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/981f2019-8639-47c8-b1d3-f2dbb02acd15-systemd-units\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.044581 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.042088    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8686a32b-9d47-4866-aae0-89f0b6575ff3-host\") pod \"node-ca-tr2kj\" (UID: \"8686a32b-9d47-4866-aae0-89f0b6575ff3\") " pod="openshift-image-registry/node-ca-tr2kj"
Sep 10 00:49:46.044581 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:46.042220    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 00:49:46.044581 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:46.042283    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00cec203-bdcf-4e75-acfa-29e1b784840e-original-pull-secret podName:00cec203-bdcf-4e75-acfa-29e1b784840e nodeName:}" failed. No retries permitted until 2026-09-10 00:49:46.542265261 +0000 UTC m=+3.039564853 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/00cec203-bdcf-4e75-acfa-29e1b784840e-original-pull-secret") pod "global-pull-secret-syncer-l4w6p" (UID: "00cec203-bdcf-4e75-acfa-29e1b784840e") : object "kube-system"/"original-pull-secret" not registered
Sep 10 00:49:46.044581 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.042472    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/86435252-3772-46f1-a9a3-c8a229064702-kubelet-dir\") pod \"aws-ebs-csi-driver-node-f27tb\" (UID: \"86435252-3772-46f1-a9a3-c8a229064702\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f27tb"
Sep 10 00:49:46.044581 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.042521    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a312259-8fb8-4bb4-9304-0b5aac8240f2-host-var-lib-kubelet\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:46.044581 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.042641    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/283dd2c2-21ef-431b-893d-b9969c7471a7-etc-sysctl-d\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:46.044581 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.042713    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/283dd2c2-21ef-431b-893d-b9969c7471a7-etc-modprobe-d\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:46.044581 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.042806    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/981f2019-8639-47c8-b1d3-f2dbb02acd15-env-overrides\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.046969 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.045808    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/283dd2c2-21ef-431b-893d-b9969c7471a7-tmp\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:46.048573 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.048548    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/981f2019-8639-47c8-b1d3-f2dbb02acd15-ovn-node-metrics-cert\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.048573 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.048563    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/283dd2c2-21ef-431b-893d-b9969c7471a7-etc-tuned\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:46.052780 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.052752    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58cp4\" (UniqueName: \"kubernetes.io/projected/283dd2c2-21ef-431b-893d-b9969c7471a7-kube-api-access-58cp4\") pod \"tuned-8qrc5\" (UID: \"283dd2c2-21ef-431b-893d-b9969c7471a7\") " pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:46.053255 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.053235    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpq6d\" (UniqueName: \"kubernetes.io/projected/7a312259-8fb8-4bb4-9304-0b5aac8240f2-kube-api-access-dpq6d\") pod \"multus-9zt49\" (UID: \"7a312259-8fb8-4bb4-9304-0b5aac8240f2\") " pod="openshift-multus/multus-9zt49"
Sep 10 00:49:46.054494 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.053694    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s52dd\" (UniqueName: \"kubernetes.io/projected/86435252-3772-46f1-a9a3-c8a229064702-kube-api-access-s52dd\") pod \"aws-ebs-csi-driver-node-f27tb\" (UID: \"86435252-3772-46f1-a9a3-c8a229064702\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f27tb"
Sep 10 00:49:46.054832 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.054785    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfxj8\" (UniqueName: \"kubernetes.io/projected/981f2019-8639-47c8-b1d3-f2dbb02acd15-kube-api-access-jfxj8\") pod \"ovnkube-node-8nwz8\" (UID: \"981f2019-8639-47c8-b1d3-f2dbb02acd15\") " pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.056186 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.056124    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4d429\" (UniqueName: \"kubernetes.io/projected/8686a32b-9d47-4866-aae0-89f0b6575ff3-kube-api-access-4d429\") pod \"node-ca-tr2kj\" (UID: \"8686a32b-9d47-4866-aae0-89f0b6575ff3\") " pod="openshift-image-registry/node-ca-tr2kj"
Sep 10 00:49:46.088534 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.088436    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-204.ec2.internal" event={"ID":"11a33808a44d7e8ec5f92874d5f9136a","Type":"ContainerStarted","Data":"629916ec8f8b47e3411169e8f1f1af1411a82875adf35ae6279325eb7cf9fbef"}
Sep 10 00:49:46.090886 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.090854    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-204.ec2.internal" event={"ID":"4ce798d5035df86fd4210aa6d5fc499e","Type":"ContainerStarted","Data":"b378934ed39a6cafa581a3a6aeb8e04f10cc6e73fa69d9b9d696a7f92c1543d1"}
Sep 10 00:49:46.142075 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.142024    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27351a7f-151c-4d9a-bddd-4ebaa3a42ade-tmp-dir\") pod \"node-resolver-p5zpt\" (UID: \"27351a7f-151c-4d9a-bddd-4ebaa3a42ade\") " pod="openshift-dns/node-resolver-p5zpt"
Sep 10 00:49:46.142263 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.142084    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/afe1ccfd-10c6-48e7-83aa-1d1384f532e8-iptables-alerter-script\") pod \"iptables-alerter-cncq7\" (UID: \"afe1ccfd-10c6-48e7-83aa-1d1384f532e8\") " pod="openshift-network-operator/iptables-alerter-cncq7"
Sep 10 00:49:46.142263 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.142115    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9f66caff-9501-4b7e-8986-9161a3ae70e3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6nzvr\" (UID: \"9f66caff-9501-4b7e-8986-9161a3ae70e3\") " pod="openshift-multus/multus-additional-cni-plugins-6nzvr"
Sep 10 00:49:46.142263 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.142143    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkfsl\" (UniqueName: \"kubernetes.io/projected/9f66caff-9501-4b7e-8986-9161a3ae70e3-kube-api-access-vkfsl\") pod \"multus-additional-cni-plugins-6nzvr\" (UID: \"9f66caff-9501-4b7e-8986-9161a3ae70e3\") " pod="openshift-multus/multus-additional-cni-plugins-6nzvr"
Sep 10 00:49:46.142263 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.142170    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9f66caff-9501-4b7e-8986-9161a3ae70e3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6nzvr\" (UID: \"9f66caff-9501-4b7e-8986-9161a3ae70e3\") " pod="openshift-multus/multus-additional-cni-plugins-6nzvr"
Sep 10 00:49:46.142263 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.142210    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/27351a7f-151c-4d9a-bddd-4ebaa3a42ade-hosts-file\") pod \"node-resolver-p5zpt\" (UID: \"27351a7f-151c-4d9a-bddd-4ebaa3a42ade\") " pod="openshift-dns/node-resolver-p5zpt"
Sep 10 00:49:46.142263 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.142236    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kctp\" (UniqueName: \"kubernetes.io/projected/50d35dde-26ab-4a1a-8f6a-2d663df04bc5-kube-api-access-2kctp\") pod \"network-check-target-cvjrx\" (UID: \"50d35dde-26ab-4a1a-8f6a-2d663df04bc5\") " pod="openshift-network-diagnostics/network-check-target-cvjrx"
Sep 10 00:49:46.142583 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.142268    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqqdw\" (UniqueName: \"kubernetes.io/projected/27351a7f-151c-4d9a-bddd-4ebaa3a42ade-kube-api-access-rqqdw\") pod \"node-resolver-p5zpt\" (UID: \"27351a7f-151c-4d9a-bddd-4ebaa3a42ade\") " pod="openshift-dns/node-resolver-p5zpt"
Sep 10 00:49:46.142583 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.142309    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/afe1ccfd-10c6-48e7-83aa-1d1384f532e8-host-slash\") pod \"iptables-alerter-cncq7\" (UID: \"afe1ccfd-10c6-48e7-83aa-1d1384f532e8\") " pod="openshift-network-operator/iptables-alerter-cncq7"
Sep 10 00:49:46.142583 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.142339    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc958bde-e5aa-468a-ae10-0c49f9438168-metrics-certs\") pod \"network-metrics-daemon-8ltcj\" (UID: \"fc958bde-e5aa-468a-ae10-0c49f9438168\") " pod="openshift-multus/network-metrics-daemon-8ltcj"
Sep 10 00:49:46.142583 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.142347    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27351a7f-151c-4d9a-bddd-4ebaa3a42ade-tmp-dir\") pod \"node-resolver-p5zpt\" (UID: \"27351a7f-151c-4d9a-bddd-4ebaa3a42ade\") " pod="openshift-dns/node-resolver-p5zpt"
Sep 10 00:49:46.142583 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.142366    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9f66caff-9501-4b7e-8986-9161a3ae70e3-system-cni-dir\") pod \"multus-additional-cni-plugins-6nzvr\" (UID: \"9f66caff-9501-4b7e-8986-9161a3ae70e3\") " pod="openshift-multus/multus-additional-cni-plugins-6nzvr"
Sep 10 00:49:46.142583 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.142394    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9f66caff-9501-4b7e-8986-9161a3ae70e3-cnibin\") pod \"multus-additional-cni-plugins-6nzvr\" (UID: \"9f66caff-9501-4b7e-8986-9161a3ae70e3\") " pod="openshift-multus/multus-additional-cni-plugins-6nzvr"
Sep 10 00:49:46.142583 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.142417    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/27351a7f-151c-4d9a-bddd-4ebaa3a42ade-hosts-file\") pod \"node-resolver-p5zpt\" (UID: \"27351a7f-151c-4d9a-bddd-4ebaa3a42ade\") " pod="openshift-dns/node-resolver-p5zpt"
Sep 10 00:49:46.142583 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.142419    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/caff9962-15c8-4c05-ad49-a748c94a3b85-konnectivity-ca\") pod \"konnectivity-agent-qhbzc\" (UID: \"caff9962-15c8-4c05-ad49-a748c94a3b85\") " pod="kube-system/konnectivity-agent-qhbzc"
Sep 10 00:49:46.142583 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.142451    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbcfv\" (UniqueName: \"kubernetes.io/projected/afe1ccfd-10c6-48e7-83aa-1d1384f532e8-kube-api-access-vbcfv\") pod \"iptables-alerter-cncq7\" (UID: \"afe1ccfd-10c6-48e7-83aa-1d1384f532e8\") " pod="openshift-network-operator/iptables-alerter-cncq7"
Sep 10 00:49:46.142583 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.142477    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9f66caff-9501-4b7e-8986-9161a3ae70e3-cni-binary-copy\") pod \"multus-additional-cni-plugins-6nzvr\" (UID: \"9f66caff-9501-4b7e-8986-9161a3ae70e3\") " pod="openshift-multus/multus-additional-cni-plugins-6nzvr"
Sep 10 00:49:46.142583 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.142505    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9f66caff-9501-4b7e-8986-9161a3ae70e3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6nzvr\" (UID: \"9f66caff-9501-4b7e-8986-9161a3ae70e3\") " pod="openshift-multus/multus-additional-cni-plugins-6nzvr"
Sep 10 00:49:46.142583 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.142532    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/caff9962-15c8-4c05-ad49-a748c94a3b85-agent-certs\") pod \"konnectivity-agent-qhbzc\" (UID: \"caff9962-15c8-4c05-ad49-a748c94a3b85\") " pod="kube-system/konnectivity-agent-qhbzc"
Sep 10 00:49:46.142583 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.142563    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8csw6\" (UniqueName: \"kubernetes.io/projected/fc958bde-e5aa-468a-ae10-0c49f9438168-kube-api-access-8csw6\") pod \"network-metrics-daemon-8ltcj\" (UID: \"fc958bde-e5aa-468a-ae10-0c49f9438168\") " pod="openshift-multus/network-metrics-daemon-8ltcj"
Sep 10 00:49:46.143296 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.142789    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/afe1ccfd-10c6-48e7-83aa-1d1384f532e8-iptables-alerter-script\") pod \"iptables-alerter-cncq7\" (UID: \"afe1ccfd-10c6-48e7-83aa-1d1384f532e8\") " pod="openshift-network-operator/iptables-alerter-cncq7"
Sep 10 00:49:46.143368 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.143340    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/afe1ccfd-10c6-48e7-83aa-1d1384f532e8-host-slash\") pod \"iptables-alerter-cncq7\" (UID: \"afe1ccfd-10c6-48e7-83aa-1d1384f532e8\") " pod="openshift-network-operator/iptables-alerter-cncq7"
Sep 10 00:49:46.143368 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.143357    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9f66caff-9501-4b7e-8986-9161a3ae70e3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6nzvr\" (UID: \"9f66caff-9501-4b7e-8986-9161a3ae70e3\") " pod="openshift-multus/multus-additional-cni-plugins-6nzvr"
Sep 10 00:49:46.143480 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.143414    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9f66caff-9501-4b7e-8986-9161a3ae70e3-system-cni-dir\") pod \"multus-additional-cni-plugins-6nzvr\" (UID: \"9f66caff-9501-4b7e-8986-9161a3ae70e3\") " pod="openshift-multus/multus-additional-cni-plugins-6nzvr"
Sep 10 00:49:46.143480 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:46.143444    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 00:49:46.143480 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.143463    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9f66caff-9501-4b7e-8986-9161a3ae70e3-cnibin\") pod \"multus-additional-cni-plugins-6nzvr\" (UID: \"9f66caff-9501-4b7e-8986-9161a3ae70e3\") " pod="openshift-multus/multus-additional-cni-plugins-6nzvr"
Sep 10 00:49:46.143631 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:46.143503    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc958bde-e5aa-468a-ae10-0c49f9438168-metrics-certs podName:fc958bde-e5aa-468a-ae10-0c49f9438168 nodeName:}" failed. No retries permitted until 2026-09-10 00:49:46.643482502 +0000 UTC m=+3.140782066 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fc958bde-e5aa-468a-ae10-0c49f9438168-metrics-certs") pod "network-metrics-daemon-8ltcj" (UID: "fc958bde-e5aa-468a-ae10-0c49f9438168") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 00:49:46.143959 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.143937    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/caff9962-15c8-4c05-ad49-a748c94a3b85-konnectivity-ca\") pod \"konnectivity-agent-qhbzc\" (UID: \"caff9962-15c8-4c05-ad49-a748c94a3b85\") " pod="kube-system/konnectivity-agent-qhbzc"
Sep 10 00:49:46.144160 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.144137    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9f66caff-9501-4b7e-8986-9161a3ae70e3-cni-binary-copy\") pod \"multus-additional-cni-plugins-6nzvr\" (UID: \"9f66caff-9501-4b7e-8986-9161a3ae70e3\") " pod="openshift-multus/multus-additional-cni-plugins-6nzvr"
Sep 10 00:49:46.144649 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.144620    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9f66caff-9501-4b7e-8986-9161a3ae70e3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6nzvr\" (UID: \"9f66caff-9501-4b7e-8986-9161a3ae70e3\") " pod="openshift-multus/multus-additional-cni-plugins-6nzvr"
Sep 10 00:49:46.145286 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.145262    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9f66caff-9501-4b7e-8986-9161a3ae70e3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6nzvr\" (UID: \"9f66caff-9501-4b7e-8986-9161a3ae70e3\") " pod="openshift-multus/multus-additional-cni-plugins-6nzvr"
Sep 10 00:49:46.148154 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.148110    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/caff9962-15c8-4c05-ad49-a748c94a3b85-agent-certs\") pod \"konnectivity-agent-qhbzc\" (UID: \"caff9962-15c8-4c05-ad49-a748c94a3b85\") " pod="kube-system/konnectivity-agent-qhbzc"
Sep 10 00:49:46.150839 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:46.150570    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 00:49:46.150839 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:46.150594    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 00:49:46.150839 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:46.150609    2563 projected.go:196] Error preparing data for projected volume kube-api-access-2kctp for pod openshift-network-diagnostics/network-check-target-cvjrx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 00:49:46.150839 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:46.150687    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50d35dde-26ab-4a1a-8f6a-2d663df04bc5-kube-api-access-2kctp podName:50d35dde-26ab-4a1a-8f6a-2d663df04bc5 nodeName:}" failed. No retries permitted until 2026-09-10 00:49:46.650668619 +0000 UTC m=+3.147968184 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2kctp" (UniqueName: "kubernetes.io/projected/50d35dde-26ab-4a1a-8f6a-2d663df04bc5-kube-api-access-2kctp") pod "network-check-target-cvjrx" (UID: "50d35dde-26ab-4a1a-8f6a-2d663df04bc5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 00:49:46.153634 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.153601    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqqdw\" (UniqueName: \"kubernetes.io/projected/27351a7f-151c-4d9a-bddd-4ebaa3a42ade-kube-api-access-rqqdw\") pod \"node-resolver-p5zpt\" (UID: \"27351a7f-151c-4d9a-bddd-4ebaa3a42ade\") " pod="openshift-dns/node-resolver-p5zpt"
Sep 10 00:49:46.153740 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.153717    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8csw6\" (UniqueName: \"kubernetes.io/projected/fc958bde-e5aa-468a-ae10-0c49f9438168-kube-api-access-8csw6\") pod \"network-metrics-daemon-8ltcj\" (UID: \"fc958bde-e5aa-468a-ae10-0c49f9438168\") " pod="openshift-multus/network-metrics-daemon-8ltcj"
Sep 10 00:49:46.154371 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.154353    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkfsl\" (UniqueName: \"kubernetes.io/projected/9f66caff-9501-4b7e-8986-9161a3ae70e3-kube-api-access-vkfsl\") pod \"multus-additional-cni-plugins-6nzvr\" (UID: \"9f66caff-9501-4b7e-8986-9161a3ae70e3\") " pod="openshift-multus/multus-additional-cni-plugins-6nzvr"
Sep 10 00:49:46.154874 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.154838    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbcfv\" (UniqueName: \"kubernetes.io/projected/afe1ccfd-10c6-48e7-83aa-1d1384f532e8-kube-api-access-vbcfv\") pod \"iptables-alerter-cncq7\" (UID: \"afe1ccfd-10c6-48e7-83aa-1d1384f532e8\") " pod="openshift-network-operator/iptables-alerter-cncq7"
Sep 10 00:49:46.223758 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.223720    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:49:46.231329 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.231300    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f27tb"
Sep 10 00:49:46.243324 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.243293    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8qrc5"
Sep 10 00:49:46.248365 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.248344    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tr2kj"
Sep 10 00:49:46.255569 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.255513    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9zt49"
Sep 10 00:49:46.262551 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.262531    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6nzvr"
Sep 10 00:49:46.277506 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.277482    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qhbzc"
Sep 10 00:49:46.286424 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.286398    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-p5zpt"
Sep 10 00:49:46.295376 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.295355    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cncq7"
Sep 10 00:49:46.547055 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.546939    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/00cec203-bdcf-4e75-acfa-29e1b784840e-original-pull-secret\") pod \"global-pull-secret-syncer-l4w6p\" (UID: \"00cec203-bdcf-4e75-acfa-29e1b784840e\") " pod="kube-system/global-pull-secret-syncer-l4w6p"
Sep 10 00:49:46.547235 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:46.547109    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 00:49:46.547235 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:46.547190    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00cec203-bdcf-4e75-acfa-29e1b784840e-original-pull-secret podName:00cec203-bdcf-4e75-acfa-29e1b784840e nodeName:}" failed. No retries permitted until 2026-09-10 00:49:47.547169602 +0000 UTC m=+4.044469180 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/00cec203-bdcf-4e75-acfa-29e1b784840e-original-pull-secret") pod "global-pull-secret-syncer-l4w6p" (UID: "00cec203-bdcf-4e75-acfa-29e1b784840e") : object "kube-system"/"original-pull-secret" not registered
Sep 10 00:49:46.647979 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.647943    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc958bde-e5aa-468a-ae10-0c49f9438168-metrics-certs\") pod \"network-metrics-daemon-8ltcj\" (UID: \"fc958bde-e5aa-468a-ae10-0c49f9438168\") " pod="openshift-multus/network-metrics-daemon-8ltcj"
Sep 10 00:49:46.648161 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:46.648097    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 00:49:46.648235 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:46.648167    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc958bde-e5aa-468a-ae10-0c49f9438168-metrics-certs podName:fc958bde-e5aa-468a-ae10-0c49f9438168 nodeName:}" failed. No retries permitted until 2026-09-10 00:49:47.648149832 +0000 UTC m=+4.145449415 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fc958bde-e5aa-468a-ae10-0c49f9438168-metrics-certs") pod "network-metrics-daemon-8ltcj" (UID: "fc958bde-e5aa-468a-ae10-0c49f9438168") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 00:49:46.749207 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.749169    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kctp\" (UniqueName: \"kubernetes.io/projected/50d35dde-26ab-4a1a-8f6a-2d663df04bc5-kube-api-access-2kctp\") pod \"network-check-target-cvjrx\" (UID: \"50d35dde-26ab-4a1a-8f6a-2d663df04bc5\") " pod="openshift-network-diagnostics/network-check-target-cvjrx"
Sep 10 00:49:46.749391 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:46.749333    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 00:49:46.749391 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:46.749352    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 00:49:46.749391 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:46.749361    2563 projected.go:196] Error preparing data for projected volume kube-api-access-2kctp for pod openshift-network-diagnostics/network-check-target-cvjrx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 00:49:46.749534 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:46.749424    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50d35dde-26ab-4a1a-8f6a-2d663df04bc5-kube-api-access-2kctp podName:50d35dde-26ab-4a1a-8f6a-2d663df04bc5 nodeName:}" failed. No retries permitted until 2026-09-10 00:49:47.749406792 +0000 UTC m=+4.246706364 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2kctp" (UniqueName: "kubernetes.io/projected/50d35dde-26ab-4a1a-8f6a-2d663df04bc5-kube-api-access-2kctp") pod "network-check-target-cvjrx" (UID: "50d35dde-26ab-4a1a-8f6a-2d663df04bc5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 00:49:46.971678 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.971570    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 00:44:46 +0000 UTC" deadline="2028-05-25 07:18:44.995470198 +0000 UTC"
Sep 10 00:49:46.971678 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:46.971602    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14958h28m58.02387192s"
Sep 10 00:49:47.076520 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:47.076490    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cvjrx"
Sep 10 00:49:47.076692 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:47.076615    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cvjrx" podUID="50d35dde-26ab-4a1a-8f6a-2d663df04bc5"
Sep 10 00:49:47.333823 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:47.333786    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27351a7f_151c_4d9a_bddd_4ebaa3a42ade.slice/crio-2f5230e3ef8a07b35f73852a0b8ba22c6d0baaa0c2de84b804d6a428b3436f49 WatchSource:0}: Error finding container 2f5230e3ef8a07b35f73852a0b8ba22c6d0baaa0c2de84b804d6a428b3436f49: Status 404 returned error can't find the container with id 2f5230e3ef8a07b35f73852a0b8ba22c6d0baaa0c2de84b804d6a428b3436f49
Sep 10 00:49:47.340865 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:47.340591    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f66caff_9501_4b7e_8986_9161a3ae70e3.slice/crio-064ebc426b61347818c912f3aa8808ab229514ebe9399c93f51a04173675bfe1 WatchSource:0}: Error finding container 064ebc426b61347818c912f3aa8808ab229514ebe9399c93f51a04173675bfe1: Status 404 returned error can't find the container with id 064ebc426b61347818c912f3aa8808ab229514ebe9399c93f51a04173675bfe1
Sep 10 00:49:47.341742 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:47.341666    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafe1ccfd_10c6_48e7_83aa_1d1384f532e8.slice/crio-c910dcfea414802d5c863b36b73722b6af1ce2135169a17f635a93fcd5f5c073 WatchSource:0}: Error finding container c910dcfea414802d5c863b36b73722b6af1ce2135169a17f635a93fcd5f5c073: Status 404 returned error can't find the container with id c910dcfea414802d5c863b36b73722b6af1ce2135169a17f635a93fcd5f5c073
Sep 10 00:49:47.343339 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:47.343235    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod283dd2c2_21ef_431b_893d_b9969c7471a7.slice/crio-f7171ebcb8c691d962f2500d31a7bb1f0999b162ee232feb7c96bb68b80a9c11 WatchSource:0}: Error finding container f7171ebcb8c691d962f2500d31a7bb1f0999b162ee232feb7c96bb68b80a9c11: Status 404 returned error can't find the container with id f7171ebcb8c691d962f2500d31a7bb1f0999b162ee232feb7c96bb68b80a9c11
Sep 10 00:49:47.344100 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:47.344013    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaff9962_15c8_4c05_ad49_a748c94a3b85.slice/crio-2d0cb5b6421817f35ced1788dc324b1df75bbf3436ec97223c0ea78036192a96 WatchSource:0}: Error finding container 2d0cb5b6421817f35ced1788dc324b1df75bbf3436ec97223c0ea78036192a96: Status 404 returned error can't find the container with id 2d0cb5b6421817f35ced1788dc324b1df75bbf3436ec97223c0ea78036192a96
Sep 10 00:49:47.345086 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:47.344728    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a312259_8fb8_4bb4_9304_0b5aac8240f2.slice/crio-f12401bac16b3aa175af5893a8383ffb8c4491f01ba47e7991174d1107eb4fbd WatchSource:0}: Error finding container f12401bac16b3aa175af5893a8383ffb8c4491f01ba47e7991174d1107eb4fbd: Status 404 returned error can't find the container with id f12401bac16b3aa175af5893a8383ffb8c4491f01ba47e7991174d1107eb4fbd
Sep 10 00:49:47.345563 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:47.345538    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86435252_3772_46f1_a9a3_c8a229064702.slice/crio-f81fad0d016abce99c068a3e594dbad4ff4e56274eb8648aa345bd85549423ad WatchSource:0}: Error finding container f81fad0d016abce99c068a3e594dbad4ff4e56274eb8648aa345bd85549423ad: Status 404 returned error can't find the container with id f81fad0d016abce99c068a3e594dbad4ff4e56274eb8648aa345bd85549423ad
Sep 10 00:49:47.348590 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:49:47.348224    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod981f2019_8639_47c8_b1d3_f2dbb02acd15.slice/crio-3d02b5aeed38f611f31f109f57185caa986af4cfe2629f71553ec05d1ff58093 WatchSource:0}: Error finding container 3d02b5aeed38f611f31f109f57185caa986af4cfe2629f71553ec05d1ff58093: Status 404 returned error can't find the container with id 3d02b5aeed38f611f31f109f57185caa986af4cfe2629f71553ec05d1ff58093
Sep 10 00:49:47.554794 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:47.554748    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/00cec203-bdcf-4e75-acfa-29e1b784840e-original-pull-secret\") pod \"global-pull-secret-syncer-l4w6p\" (UID: \"00cec203-bdcf-4e75-acfa-29e1b784840e\") " pod="kube-system/global-pull-secret-syncer-l4w6p"
Sep 10 00:49:47.554987 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:47.554877    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 00:49:47.554987 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:47.554946    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00cec203-bdcf-4e75-acfa-29e1b784840e-original-pull-secret podName:00cec203-bdcf-4e75-acfa-29e1b784840e nodeName:}" failed. No retries permitted until 2026-09-10 00:49:49.554927289 +0000 UTC m=+6.052226855 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/00cec203-bdcf-4e75-acfa-29e1b784840e-original-pull-secret") pod "global-pull-secret-syncer-l4w6p" (UID: "00cec203-bdcf-4e75-acfa-29e1b784840e") : object "kube-system"/"original-pull-secret" not registered
Sep 10 00:49:47.655226 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:47.655197    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc958bde-e5aa-468a-ae10-0c49f9438168-metrics-certs\") pod \"network-metrics-daemon-8ltcj\" (UID: \"fc958bde-e5aa-468a-ae10-0c49f9438168\") " pod="openshift-multus/network-metrics-daemon-8ltcj"
Sep 10 00:49:47.655337 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:47.655310    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 00:49:47.655378 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:47.655360    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc958bde-e5aa-468a-ae10-0c49f9438168-metrics-certs podName:fc958bde-e5aa-468a-ae10-0c49f9438168 nodeName:}" failed. No retries permitted until 2026-09-10 00:49:49.655345978 +0000 UTC m=+6.152645554 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fc958bde-e5aa-468a-ae10-0c49f9438168-metrics-certs") pod "network-metrics-daemon-8ltcj" (UID: "fc958bde-e5aa-468a-ae10-0c49f9438168") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 00:49:47.755526 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:47.755473    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kctp\" (UniqueName: \"kubernetes.io/projected/50d35dde-26ab-4a1a-8f6a-2d663df04bc5-kube-api-access-2kctp\") pod \"network-check-target-cvjrx\" (UID: \"50d35dde-26ab-4a1a-8f6a-2d663df04bc5\") " pod="openshift-network-diagnostics/network-check-target-cvjrx"
Sep 10 00:49:47.755690 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:47.755650    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 00:49:47.755690 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:47.755671    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 00:49:47.755690 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:47.755683    2563 projected.go:196] Error preparing data for projected volume kube-api-access-2kctp for pod openshift-network-diagnostics/network-check-target-cvjrx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 00:49:47.755810 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:47.755739    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50d35dde-26ab-4a1a-8f6a-2d663df04bc5-kube-api-access-2kctp podName:50d35dde-26ab-4a1a-8f6a-2d663df04bc5 nodeName:}" failed. No retries permitted until 2026-09-10 00:49:49.755724689 +0000 UTC m=+6.253024248 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2kctp" (UniqueName: "kubernetes.io/projected/50d35dde-26ab-4a1a-8f6a-2d663df04bc5-kube-api-access-2kctp") pod "network-check-target-cvjrx" (UID: "50d35dde-26ab-4a1a-8f6a-2d663df04bc5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 00:49:47.972015 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:47.971912    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 00:44:46 +0000 UTC" deadline="2028-04-30 23:02:31.620882465 +0000 UTC"
Sep 10 00:49:47.972015 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:47.971981    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14374h12m43.648905278s"
Sep 10 00:49:48.079121 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:48.076335    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltcj"
Sep 10 00:49:48.079121 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:48.076494    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8ltcj" podUID="fc958bde-e5aa-468a-ae10-0c49f9438168"
Sep 10 00:49:48.079121 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:48.077050    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l4w6p"
Sep 10 00:49:48.079121 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:48.077170    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l4w6p" podUID="00cec203-bdcf-4e75-acfa-29e1b784840e"
Sep 10 00:49:48.101733 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:48.101684    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cncq7" event={"ID":"afe1ccfd-10c6-48e7-83aa-1d1384f532e8","Type":"ContainerStarted","Data":"c910dcfea414802d5c863b36b73722b6af1ce2135169a17f635a93fcd5f5c073"}
Sep 10 00:49:48.104422 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:48.104389    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qhbzc" event={"ID":"caff9962-15c8-4c05-ad49-a748c94a3b85","Type":"ContainerStarted","Data":"2d0cb5b6421817f35ced1788dc324b1df75bbf3436ec97223c0ea78036192a96"}
Sep 10 00:49:48.106799 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:48.106769    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8qrc5" event={"ID":"283dd2c2-21ef-431b-893d-b9969c7471a7","Type":"ContainerStarted","Data":"f7171ebcb8c691d962f2500d31a7bb1f0999b162ee232feb7c96bb68b80a9c11"}
Sep 10 00:49:48.111434 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:48.111407    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tr2kj" event={"ID":"8686a32b-9d47-4866-aae0-89f0b6575ff3","Type":"ContainerStarted","Data":"3aabcb3955f33c09e3775070d948a73ff6915d10cb74491e4399a63e453ca83f"}
Sep 10 00:49:48.114603 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:48.114576    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-204.ec2.internal" event={"ID":"4ce798d5035df86fd4210aa6d5fc499e","Type":"ContainerStarted","Data":"2cb28805fd84b5319c31b03dac04b1281528afdf87289b5be531ae9125f8fbc6"}
Sep 10 00:49:48.117192 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:48.117167    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f27tb" event={"ID":"86435252-3772-46f1-a9a3-c8a229064702","Type":"ContainerStarted","Data":"f81fad0d016abce99c068a3e594dbad4ff4e56274eb8648aa345bd85549423ad"}
Sep 10 00:49:48.120156 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:48.120131    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8" event={"ID":"981f2019-8639-47c8-b1d3-f2dbb02acd15","Type":"ContainerStarted","Data":"3d02b5aeed38f611f31f109f57185caa986af4cfe2629f71553ec05d1ff58093"}
Sep 10 00:49:48.122161 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:48.122132    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9zt49" event={"ID":"7a312259-8fb8-4bb4-9304-0b5aac8240f2","Type":"ContainerStarted","Data":"f12401bac16b3aa175af5893a8383ffb8c4491f01ba47e7991174d1107eb4fbd"}
Sep 10 00:49:48.126552 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:48.126527    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6nzvr" event={"ID":"9f66caff-9501-4b7e-8986-9161a3ae70e3","Type":"ContainerStarted","Data":"064ebc426b61347818c912f3aa8808ab229514ebe9399c93f51a04173675bfe1"}
Sep 10 00:49:48.130287 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:48.130262    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-p5zpt" event={"ID":"27351a7f-151c-4d9a-bddd-4ebaa3a42ade","Type":"ContainerStarted","Data":"2f5230e3ef8a07b35f73852a0b8ba22c6d0baaa0c2de84b804d6a428b3436f49"}
Sep 10 00:49:48.139362 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:48.138328    2563 generic.go:356] "Generic (PLEG): container finished" podID="11a33808a44d7e8ec5f92874d5f9136a" containerID="9cb9056db8ddd2412ad97cc065f5d91edec1f03fdd417095cf7b491253e77b6f" exitCode=0
Sep 10 00:49:48.139362 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:48.138364    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-204.ec2.internal" event={"ID":"11a33808a44d7e8ec5f92874d5f9136a","Type":"ContainerDied","Data":"9cb9056db8ddd2412ad97cc065f5d91edec1f03fdd417095cf7b491253e77b6f"}
Sep 10 00:49:48.156451 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:48.156385    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-204.ec2.internal" podStartSLOduration=3.156363296 podStartE2EDuration="3.156363296s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 00:49:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:49:48.133609332 +0000 UTC m=+4.630908916" watchObservedRunningTime="2026-09-10 00:49:48.156363296 +0000 UTC m=+4.653662881"
Sep 10 00:49:49.077483 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:49.076840    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cvjrx"
Sep 10 00:49:49.077483 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:49.076978    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cvjrx" podUID="50d35dde-26ab-4a1a-8f6a-2d663df04bc5"
Sep 10 00:49:49.162493 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:49.162415    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-204.ec2.internal" event={"ID":"11a33808a44d7e8ec5f92874d5f9136a","Type":"ContainerStarted","Data":"e650dae03479dad06ce9b8456af5a005276f4309149b5cba0c5b6071be002caa"}
Sep 10 00:49:49.571666 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:49.571568    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/00cec203-bdcf-4e75-acfa-29e1b784840e-original-pull-secret\") pod \"global-pull-secret-syncer-l4w6p\" (UID: \"00cec203-bdcf-4e75-acfa-29e1b784840e\") " pod="kube-system/global-pull-secret-syncer-l4w6p"
Sep 10 00:49:49.571838 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:49.571762    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 00:49:49.571838 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:49.571831    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00cec203-bdcf-4e75-acfa-29e1b784840e-original-pull-secret podName:00cec203-bdcf-4e75-acfa-29e1b784840e nodeName:}" failed. No retries permitted until 2026-09-10 00:49:53.571810561 +0000 UTC m=+10.069110142 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/00cec203-bdcf-4e75-acfa-29e1b784840e-original-pull-secret") pod "global-pull-secret-syncer-l4w6p" (UID: "00cec203-bdcf-4e75-acfa-29e1b784840e") : object "kube-system"/"original-pull-secret" not registered
Sep 10 00:49:49.672787 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:49.672748    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc958bde-e5aa-468a-ae10-0c49f9438168-metrics-certs\") pod \"network-metrics-daemon-8ltcj\" (UID: \"fc958bde-e5aa-468a-ae10-0c49f9438168\") " pod="openshift-multus/network-metrics-daemon-8ltcj"
Sep 10 00:49:49.672995 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:49.672913    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 00:49:49.672995 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:49.672982    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc958bde-e5aa-468a-ae10-0c49f9438168-metrics-certs podName:fc958bde-e5aa-468a-ae10-0c49f9438168 nodeName:}" failed. No retries permitted until 2026-09-10 00:49:53.67296087 +0000 UTC m=+10.170260432 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fc958bde-e5aa-468a-ae10-0c49f9438168-metrics-certs") pod "network-metrics-daemon-8ltcj" (UID: "fc958bde-e5aa-468a-ae10-0c49f9438168") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 00:49:49.774598 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:49.773874    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kctp\" (UniqueName: \"kubernetes.io/projected/50d35dde-26ab-4a1a-8f6a-2d663df04bc5-kube-api-access-2kctp\") pod \"network-check-target-cvjrx\" (UID: \"50d35dde-26ab-4a1a-8f6a-2d663df04bc5\") " pod="openshift-network-diagnostics/network-check-target-cvjrx"
Sep 10 00:49:49.774598 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:49.774116    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 00:49:49.774598 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:49.774140    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 00:49:49.774598 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:49.774154    2563 projected.go:196] Error preparing data for projected volume kube-api-access-2kctp for pod openshift-network-diagnostics/network-check-target-cvjrx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 00:49:49.774598 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:49.774215    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50d35dde-26ab-4a1a-8f6a-2d663df04bc5-kube-api-access-2kctp podName:50d35dde-26ab-4a1a-8f6a-2d663df04bc5 nodeName:}" failed. No retries permitted until 2026-09-10 00:49:53.774196155 +0000 UTC m=+10.271495719 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2kctp" (UniqueName: "kubernetes.io/projected/50d35dde-26ab-4a1a-8f6a-2d663df04bc5-kube-api-access-2kctp") pod "network-check-target-cvjrx" (UID: "50d35dde-26ab-4a1a-8f6a-2d663df04bc5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 00:49:50.079364 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:50.076165    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltcj"
Sep 10 00:49:50.079364 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:50.076323    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8ltcj" podUID="fc958bde-e5aa-468a-ae10-0c49f9438168"
Sep 10 00:49:50.079364 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:50.076397    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l4w6p"
Sep 10 00:49:50.079364 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:50.076472    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l4w6p" podUID="00cec203-bdcf-4e75-acfa-29e1b784840e"
Sep 10 00:49:51.076739 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:51.076705    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cvjrx"
Sep 10 00:49:51.076904 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:51.076854    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cvjrx" podUID="50d35dde-26ab-4a1a-8f6a-2d663df04bc5"
Sep 10 00:49:52.080917 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:52.080883    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l4w6p"
Sep 10 00:49:52.081456 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:52.081019    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l4w6p" podUID="00cec203-bdcf-4e75-acfa-29e1b784840e"
Sep 10 00:49:52.081535 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:52.081471    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltcj"
Sep 10 00:49:52.081626 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:52.081594    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8ltcj" podUID="fc958bde-e5aa-468a-ae10-0c49f9438168"
Sep 10 00:49:53.076298 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:53.076260    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cvjrx"
Sep 10 00:49:53.076514 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:53.076403    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cvjrx" podUID="50d35dde-26ab-4a1a-8f6a-2d663df04bc5"
Sep 10 00:49:53.608520 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:53.608479    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/00cec203-bdcf-4e75-acfa-29e1b784840e-original-pull-secret\") pod \"global-pull-secret-syncer-l4w6p\" (UID: \"00cec203-bdcf-4e75-acfa-29e1b784840e\") " pod="kube-system/global-pull-secret-syncer-l4w6p"
Sep 10 00:49:53.608987 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:53.608665    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 00:49:53.608987 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:53.608743    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00cec203-bdcf-4e75-acfa-29e1b784840e-original-pull-secret podName:00cec203-bdcf-4e75-acfa-29e1b784840e nodeName:}" failed. No retries permitted until 2026-09-10 00:50:01.608721095 +0000 UTC m=+18.106020672 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/00cec203-bdcf-4e75-acfa-29e1b784840e-original-pull-secret") pod "global-pull-secret-syncer-l4w6p" (UID: "00cec203-bdcf-4e75-acfa-29e1b784840e") : object "kube-system"/"original-pull-secret" not registered
Sep 10 00:49:53.709069 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:53.709030    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc958bde-e5aa-468a-ae10-0c49f9438168-metrics-certs\") pod \"network-metrics-daemon-8ltcj\" (UID: \"fc958bde-e5aa-468a-ae10-0c49f9438168\") " pod="openshift-multus/network-metrics-daemon-8ltcj"
Sep 10 00:49:53.709428 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:53.709209    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 00:49:53.709428 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:53.709279    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc958bde-e5aa-468a-ae10-0c49f9438168-metrics-certs podName:fc958bde-e5aa-468a-ae10-0c49f9438168 nodeName:}" failed. No retries permitted until 2026-09-10 00:50:01.709258466 +0000 UTC m=+18.206558047 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fc958bde-e5aa-468a-ae10-0c49f9438168-metrics-certs") pod "network-metrics-daemon-8ltcj" (UID: "fc958bde-e5aa-468a-ae10-0c49f9438168") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 00:49:53.810457 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:53.810415    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kctp\" (UniqueName: \"kubernetes.io/projected/50d35dde-26ab-4a1a-8f6a-2d663df04bc5-kube-api-access-2kctp\") pod \"network-check-target-cvjrx\" (UID: \"50d35dde-26ab-4a1a-8f6a-2d663df04bc5\") " pod="openshift-network-diagnostics/network-check-target-cvjrx"
Sep 10 00:49:53.810643 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:53.810596    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 00:49:53.810643 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:53.810614    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 00:49:53.810643 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:53.810627    2563 projected.go:196] Error preparing data for projected volume kube-api-access-2kctp for pod openshift-network-diagnostics/network-check-target-cvjrx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 00:49:53.810824 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:53.810683    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50d35dde-26ab-4a1a-8f6a-2d663df04bc5-kube-api-access-2kctp podName:50d35dde-26ab-4a1a-8f6a-2d663df04bc5 nodeName:}" failed. No retries permitted until 2026-09-10 00:50:01.810664558 +0000 UTC m=+18.307964137 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2kctp" (UniqueName: "kubernetes.io/projected/50d35dde-26ab-4a1a-8f6a-2d663df04bc5-kube-api-access-2kctp") pod "network-check-target-cvjrx" (UID: "50d35dde-26ab-4a1a-8f6a-2d663df04bc5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 00:49:54.080651 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:54.080571    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltcj"
Sep 10 00:49:54.080808 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:54.080691    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8ltcj" podUID="fc958bde-e5aa-468a-ae10-0c49f9438168"
Sep 10 00:49:54.080808 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:54.080749    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l4w6p"
Sep 10 00:49:54.080934 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:54.080867    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l4w6p" podUID="00cec203-bdcf-4e75-acfa-29e1b784840e"
Sep 10 00:49:55.076877 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:55.076831    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cvjrx"
Sep 10 00:49:55.077486 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:55.076993    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cvjrx" podUID="50d35dde-26ab-4a1a-8f6a-2d663df04bc5"
Sep 10 00:49:56.076522 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:56.076432    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l4w6p"
Sep 10 00:49:56.076678 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:56.076441    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltcj"
Sep 10 00:49:56.076678 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:56.076590    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l4w6p" podUID="00cec203-bdcf-4e75-acfa-29e1b784840e"
Sep 10 00:49:56.076678 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:56.076648    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8ltcj" podUID="fc958bde-e5aa-468a-ae10-0c49f9438168"
Sep 10 00:49:57.076488 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:57.076452    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cvjrx"
Sep 10 00:49:57.076964 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:57.076591    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cvjrx" podUID="50d35dde-26ab-4a1a-8f6a-2d663df04bc5"
Sep 10 00:49:58.076204 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:58.076148    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l4w6p"
Sep 10 00:49:58.076364 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:58.076166    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltcj"
Sep 10 00:49:58.076364 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:58.076306    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l4w6p" podUID="00cec203-bdcf-4e75-acfa-29e1b784840e"
Sep 10 00:49:58.076440 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:58.076400    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8ltcj" podUID="fc958bde-e5aa-468a-ae10-0c49f9438168"
Sep 10 00:49:59.076630 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:49:59.076551    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cvjrx"
Sep 10 00:49:59.077107 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:49:59.076678    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cvjrx" podUID="50d35dde-26ab-4a1a-8f6a-2d663df04bc5"
Sep 10 00:50:00.076182 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:00.076146    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l4w6p"
Sep 10 00:50:00.076538 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:00.076513    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltcj"
Sep 10 00:50:00.076701 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:00.076678    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8ltcj" podUID="fc958bde-e5aa-468a-ae10-0c49f9438168"
Sep 10 00:50:00.079268 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:00.077282    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l4w6p" podUID="00cec203-bdcf-4e75-acfa-29e1b784840e"
Sep 10 00:50:01.076142 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:01.075916    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cvjrx"
Sep 10 00:50:01.076373 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:01.076234    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cvjrx" podUID="50d35dde-26ab-4a1a-8f6a-2d663df04bc5"
Sep 10 00:50:01.673912 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:01.673874    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/00cec203-bdcf-4e75-acfa-29e1b784840e-original-pull-secret\") pod \"global-pull-secret-syncer-l4w6p\" (UID: \"00cec203-bdcf-4e75-acfa-29e1b784840e\") " pod="kube-system/global-pull-secret-syncer-l4w6p"
Sep 10 00:50:01.674471 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:01.674064    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 00:50:01.674471 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:01.674142    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00cec203-bdcf-4e75-acfa-29e1b784840e-original-pull-secret podName:00cec203-bdcf-4e75-acfa-29e1b784840e nodeName:}" failed. No retries permitted until 2026-09-10 00:50:17.674122238 +0000 UTC m=+34.171421795 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/00cec203-bdcf-4e75-acfa-29e1b784840e-original-pull-secret") pod "global-pull-secret-syncer-l4w6p" (UID: "00cec203-bdcf-4e75-acfa-29e1b784840e") : object "kube-system"/"original-pull-secret" not registered
Sep 10 00:50:01.775060 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:01.775028    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc958bde-e5aa-468a-ae10-0c49f9438168-metrics-certs\") pod \"network-metrics-daemon-8ltcj\" (UID: \"fc958bde-e5aa-468a-ae10-0c49f9438168\") " pod="openshift-multus/network-metrics-daemon-8ltcj"
Sep 10 00:50:01.775216 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:01.775148    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 00:50:01.775216 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:01.775205    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc958bde-e5aa-468a-ae10-0c49f9438168-metrics-certs podName:fc958bde-e5aa-468a-ae10-0c49f9438168 nodeName:}" failed. No retries permitted until 2026-09-10 00:50:17.775187531 +0000 UTC m=+34.272487108 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fc958bde-e5aa-468a-ae10-0c49f9438168-metrics-certs") pod "network-metrics-daemon-8ltcj" (UID: "fc958bde-e5aa-468a-ae10-0c49f9438168") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 00:50:01.875863 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:01.875825    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kctp\" (UniqueName: \"kubernetes.io/projected/50d35dde-26ab-4a1a-8f6a-2d663df04bc5-kube-api-access-2kctp\") pod \"network-check-target-cvjrx\" (UID: \"50d35dde-26ab-4a1a-8f6a-2d663df04bc5\") " pod="openshift-network-diagnostics/network-check-target-cvjrx"
Sep 10 00:50:01.876054 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:01.875995    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 00:50:01.876054 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:01.876013    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 00:50:01.876054 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:01.876025    2563 projected.go:196] Error preparing data for projected volume kube-api-access-2kctp for pod openshift-network-diagnostics/network-check-target-cvjrx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 00:50:01.876258 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:01.876082    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50d35dde-26ab-4a1a-8f6a-2d663df04bc5-kube-api-access-2kctp podName:50d35dde-26ab-4a1a-8f6a-2d663df04bc5 nodeName:}" failed. No retries permitted until 2026-09-10 00:50:17.876065591 +0000 UTC m=+34.373365147 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2kctp" (UniqueName: "kubernetes.io/projected/50d35dde-26ab-4a1a-8f6a-2d663df04bc5-kube-api-access-2kctp") pod "network-check-target-cvjrx" (UID: "50d35dde-26ab-4a1a-8f6a-2d663df04bc5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 00:50:02.076693 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:02.076659    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l4w6p"
Sep 10 00:50:02.076877 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:02.076689    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltcj"
Sep 10 00:50:02.076877 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:02.076819    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l4w6p" podUID="00cec203-bdcf-4e75-acfa-29e1b784840e"
Sep 10 00:50:02.077014 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:02.076937    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8ltcj" podUID="fc958bde-e5aa-468a-ae10-0c49f9438168"
Sep 10 00:50:03.076579 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:03.076548    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cvjrx"
Sep 10 00:50:03.077028 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:03.076677    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cvjrx" podUID="50d35dde-26ab-4a1a-8f6a-2d663df04bc5"
Sep 10 00:50:04.077289 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:04.077255    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l4w6p"
Sep 10 00:50:04.077753 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:04.077364    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l4w6p" podUID="00cec203-bdcf-4e75-acfa-29e1b784840e"
Sep 10 00:50:04.077753 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:04.077436    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltcj"
Sep 10 00:50:04.077753 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:04.077526    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8ltcj" podUID="fc958bde-e5aa-468a-ae10-0c49f9438168"
Sep 10 00:50:05.076396 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:05.076359    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cvjrx"
Sep 10 00:50:05.076593 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:05.076485    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cvjrx" podUID="50d35dde-26ab-4a1a-8f6a-2d663df04bc5"
Sep 10 00:50:06.076469 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:06.076446    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l4w6p"
Sep 10 00:50:06.077022 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:06.076546    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l4w6p" podUID="00cec203-bdcf-4e75-acfa-29e1b784840e"
Sep 10 00:50:06.077022 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:06.076600    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltcj"
Sep 10 00:50:06.077022 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:06.076706    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8ltcj" podUID="fc958bde-e5aa-468a-ae10-0c49f9438168"
Sep 10 00:50:06.214035 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:06.213976    2563 generic.go:356] "Generic (PLEG): container finished" podID="9f66caff-9501-4b7e-8986-9161a3ae70e3" containerID="ff3d4d35b3453436a93c563d0977c709cc1195f33c9dced39fed8c485d3c4d4e" exitCode=0
Sep 10 00:50:06.214240 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:06.214089    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6nzvr" event={"ID":"9f66caff-9501-4b7e-8986-9161a3ae70e3","Type":"ContainerDied","Data":"ff3d4d35b3453436a93c563d0977c709cc1195f33c9dced39fed8c485d3c4d4e"}
Sep 10 00:50:06.215809 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:06.215787    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-p5zpt" event={"ID":"27351a7f-151c-4d9a-bddd-4ebaa3a42ade","Type":"ContainerStarted","Data":"c71e642fc02b949362c2d43df1ee0bca91af83d412ece0f125ad2725891b3af2"}
Sep 10 00:50:06.217280 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:06.217240    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qhbzc" event={"ID":"caff9962-15c8-4c05-ad49-a748c94a3b85","Type":"ContainerStarted","Data":"10bdddc476dac0dab20ebe2030c6212aa1b504370017a568331a944aab48e76a"}
Sep 10 00:50:06.218969 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:06.218949    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8qrc5" event={"ID":"283dd2c2-21ef-431b-893d-b9969c7471a7","Type":"ContainerStarted","Data":"b67dea707081196d8bb890d04537ae5a4e8653ea5d63d051396ca875f5acfa96"}
Sep 10 00:50:06.220564 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:06.220544    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tr2kj" event={"ID":"8686a32b-9d47-4866-aae0-89f0b6575ff3","Type":"ContainerStarted","Data":"dfd56cb29f6cd7014628195ae1f514e203a261905f4571d8aca0b15b88437906"}
Sep 10 00:50:06.222083 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:06.222064    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f27tb" event={"ID":"86435252-3772-46f1-a9a3-c8a229064702","Type":"ContainerStarted","Data":"854050061a63211886cd518cfc618563f6d3d23322ba147ec93c65c0c770e4fb"}
Sep 10 00:50:06.225157 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:06.225134    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8" event={"ID":"981f2019-8639-47c8-b1d3-f2dbb02acd15","Type":"ContainerStarted","Data":"0209b53be8420f7fdd015933cec856bf187488a33ddaff760e5bacfcccbb0a92"}
Sep 10 00:50:06.225255 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:06.225215    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8" event={"ID":"981f2019-8639-47c8-b1d3-f2dbb02acd15","Type":"ContainerStarted","Data":"c3961b26fe52083cd697374f73d27e2e0349aeded6694a9755b1c4b0f70ee0e9"}
Sep 10 00:50:06.225305 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:06.225264    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8" event={"ID":"981f2019-8639-47c8-b1d3-f2dbb02acd15","Type":"ContainerStarted","Data":"f85af3422c3591d4f48bfe9e60dc4d1e57f359cbf5785eb545219a42bb8e5730"}
Sep 10 00:50:06.225347 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:06.225307    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8" event={"ID":"981f2019-8639-47c8-b1d3-f2dbb02acd15","Type":"ContainerStarted","Data":"f6d8cbaf4f21f0c9fea990d161e76eeb1c4844735abd028e1783f367520fc32d"}
Sep 10 00:50:06.225406 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:06.225393    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8" event={"ID":"981f2019-8639-47c8-b1d3-f2dbb02acd15","Type":"ContainerStarted","Data":"cd8a91b700303223ff982a1d26c526bb4189a1daa70b3fc936f65ba22acadf3b"}
Sep 10 00:50:06.225447 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:06.225441    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8" event={"ID":"981f2019-8639-47c8-b1d3-f2dbb02acd15","Type":"ContainerStarted","Data":"9f6af41033cdc6eb01452651495a664a7933cf03eb3d552a7d30cc755c2652cf"}
Sep 10 00:50:06.226614 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:06.226588    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9zt49" event={"ID":"7a312259-8fb8-4bb4-9304-0b5aac8240f2","Type":"ContainerStarted","Data":"f16d46a846c81dba35387d90162695706333b2ddcb4b4ab82ecaa819598e05bb"}
Sep 10 00:50:06.237723 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:06.237686    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-204.ec2.internal" podStartSLOduration=21.237673097 podStartE2EDuration="21.237673097s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 00:49:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:49:49.180119499 +0000 UTC m=+5.677419083" watchObservedRunningTime="2026-09-10 00:50:06.237673097 +0000 UTC m=+22.734972675"
Sep 10 00:50:06.252758 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:06.252712    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8qrc5" podStartSLOduration=2.987555566 podStartE2EDuration="21.252697897s" totalImagesPullingTime="18.265142331s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 00:49:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:50:06.252367212 +0000 UTC m=+22.749666791" watchObservedRunningTime="2026-09-10 00:50:06.252697897 +0000 UTC m=+22.749997476"
Sep 10 00:50:06.265378 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:06.265325    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-tr2kj" podStartSLOduration=10.033752772 podStartE2EDuration="21.265311055s" totalImagesPullingTime="11.231558283s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 00:49:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:50:06.264912569 +0000 UTC m=+22.762212125" watchObservedRunningTime="2026-09-10 00:50:06.265311055 +0000 UTC m=+22.762610644"
Sep 10 00:50:06.309944 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:06.309897    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-9zt49" podStartSLOduration=2.986698418 podStartE2EDuration="21.309882366s" totalImagesPullingTime="18.323183948s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 00:49:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:50:06.291293038 +0000 UTC m=+22.788592617" watchObservedRunningTime="2026-09-10 00:50:06.309882366 +0000 UTC m=+22.807181945"
Sep 10 00:50:06.310142 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:06.310077    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qhbzc" podStartSLOduration=3.048131192 podStartE2EDuration="21.310072287s" totalImagesPullingTime="18.261941095s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:49:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:50:06.309740861 +0000 UTC m=+22.807040439" watchObservedRunningTime="2026-09-10 00:50:06.310072287 +0000 UTC m=+22.807371896"
Sep 10 00:50:06.324635 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:06.324585    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-p5zpt" podStartSLOduration=3.091368537 podStartE2EDuration="21.324568216s" totalImagesPullingTime="18.233199679s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 00:49:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:50:06.324522447 +0000 UTC m=+22.821822025" watchObservedRunningTime="2026-09-10 00:50:06.324568216 +0000 UTC m=+22.821867794"
Sep 10 00:50:07.076692 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:07.076658    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cvjrx"
Sep 10 00:50:07.077371 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:07.076778    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cvjrx" podUID="50d35dde-26ab-4a1a-8f6a-2d663df04bc5"
Sep 10 00:50:07.229582 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:07.229546    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cncq7" event={"ID":"afe1ccfd-10c6-48e7-83aa-1d1384f532e8","Type":"ContainerStarted","Data":"b63748fad8dde7b4af2c3a6d86684ed6bf4840b8303b604c4846fccfcdb75d08"}
Sep 10 00:50:07.244862 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:07.244807    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-cncq7" podStartSLOduration=3.981315758 podStartE2EDuration="22.244786411s" totalImagesPullingTime="18.263470653s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 00:49:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:50:07.24439212 +0000 UTC m=+23.741691701" watchObservedRunningTime="2026-09-10 00:50:07.244786411 +0000 UTC m=+23.742085991"
Sep 10 00:50:07.340951 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:07.340921    2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 10 00:50:08.076386 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:08.076350    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltcj"
Sep 10 00:50:08.076571 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:08.076348    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l4w6p"
Sep 10 00:50:08.076571 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:08.076496    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8ltcj" podUID="fc958bde-e5aa-468a-ae10-0c49f9438168"
Sep 10 00:50:08.076571 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:08.076552    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l4w6p" podUID="00cec203-bdcf-4e75-acfa-29e1b784840e"
Sep 10 00:50:08.108023 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:08.107911    2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T00:50:07.340939623Z","UUID":"3fa23a5e-5185-4a3c-a376-36a869e9c08e","Handler":null,"Name":"","Endpoint":""}
Sep 10 00:50:08.109905 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:08.109867    2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 10 00:50:08.109905 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:08.109905    2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 10 00:50:08.234525 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:08.234336    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f27tb" event={"ID":"86435252-3772-46f1-a9a3-c8a229064702","Type":"ContainerStarted","Data":"332b10b3f22bfc233827da042dec928e38701220e00fb98cf4b28c322f81d690"}
Sep 10 00:50:09.076118 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:09.076084    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cvjrx"
Sep 10 00:50:09.076291 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:09.076197    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cvjrx" podUID="50d35dde-26ab-4a1a-8f6a-2d663df04bc5"
Sep 10 00:50:09.094791 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:09.094755    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qhbzc"
Sep 10 00:50:09.095431 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:09.095408    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qhbzc"
Sep 10 00:50:09.241666 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:09.241619    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8" event={"ID":"981f2019-8639-47c8-b1d3-f2dbb02acd15","Type":"ContainerStarted","Data":"b69c8df172845f683758540a4af24af195315a7d912f9dee1d085d8466c24327"}
Sep 10 00:50:09.242150 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:09.241722    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qhbzc"
Sep 10 00:50:09.242384 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:09.242368    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qhbzc"
Sep 10 00:50:10.076951 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:10.076856    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltcj"
Sep 10 00:50:10.077139 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:10.076860    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l4w6p"
Sep 10 00:50:10.077139 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:10.076974    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8ltcj" podUID="fc958bde-e5aa-468a-ae10-0c49f9438168"
Sep 10 00:50:10.077139 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:10.077079    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l4w6p" podUID="00cec203-bdcf-4e75-acfa-29e1b784840e"
Sep 10 00:50:10.246640 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:10.246596    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f27tb" event={"ID":"86435252-3772-46f1-a9a3-c8a229064702","Type":"ContainerStarted","Data":"58eb086339e41f5a3c12d1c2274afb92532eb103cea10fc807efa3e66942c156"}
Sep 10 00:50:10.276767 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:10.276712    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f27tb" podStartSLOduration=3.350114071 podStartE2EDuration="25.276695257s" totalImagesPullingTime="21.926581186s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 00:49:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:50:10.275253601 +0000 UTC m=+26.772553184" watchObservedRunningTime="2026-09-10 00:50:10.276695257 +0000 UTC m=+26.773994837"
Sep 10 00:50:11.077105 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:11.076880    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cvjrx"
Sep 10 00:50:11.077258 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:11.077192    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cvjrx" podUID="50d35dde-26ab-4a1a-8f6a-2d663df04bc5"
Sep 10 00:50:11.252794 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:11.252749    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8" event={"ID":"981f2019-8639-47c8-b1d3-f2dbb02acd15","Type":"ContainerStarted","Data":"556cc9d23e90a508f17e79f4772b0575870a7c4408abd6d8d45d0fb59e9e5730"}
Sep 10 00:50:11.279922 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:11.279870    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8" podStartSLOduration=7.959551108 podStartE2EDuration="26.279853025s" totalImagesPullingTime="18.320301917s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 00:49:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:50:11.278226767 +0000 UTC m=+27.775526346" watchObservedRunningTime="2026-09-10 00:50:11.279853025 +0000 UTC m=+27.777152604"
Sep 10 00:50:12.076973 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:12.076870    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltcj"
Sep 10 00:50:12.077167 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:12.077035    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8ltcj" podUID="fc958bde-e5aa-468a-ae10-0c49f9438168"
Sep 10 00:50:12.077167 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:12.077095    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l4w6p"
Sep 10 00:50:12.077271 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:12.077181    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l4w6p" podUID="00cec203-bdcf-4e75-acfa-29e1b784840e"
Sep 10 00:50:12.255181 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:12.255152    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:50:12.255626 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:12.255228    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:50:12.255626 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:12.255260    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:50:12.270281 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:12.270254    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:50:12.270430 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:12.270362    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:50:12.579617 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:12.579580    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-l4w6p"]
Sep 10 00:50:12.579783 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:12.579758    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l4w6p"
Sep 10 00:50:12.579872 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:12.579840    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l4w6p" podUID="00cec203-bdcf-4e75-acfa-29e1b784840e"
Sep 10 00:50:12.586424 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:12.586373    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cvjrx"]
Sep 10 00:50:12.586579 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:12.586514    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cvjrx"
Sep 10 00:50:12.586687 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:12.586657    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cvjrx" podUID="50d35dde-26ab-4a1a-8f6a-2d663df04bc5"
Sep 10 00:50:12.587092 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:12.587068    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8ltcj"]
Sep 10 00:50:12.587193 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:12.587181    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltcj"
Sep 10 00:50:12.587309 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:12.587287    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8ltcj" podUID="fc958bde-e5aa-468a-ae10-0c49f9438168"
Sep 10 00:50:13.258323 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:13.258290    2563 generic.go:356] "Generic (PLEG): container finished" podID="9f66caff-9501-4b7e-8986-9161a3ae70e3" containerID="295903563b9b067361e43f861a075e8e290d2485d4cdbc9531321c95f473281d" exitCode=0
Sep 10 00:50:13.259141 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:13.258377    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6nzvr" event={"ID":"9f66caff-9501-4b7e-8986-9161a3ae70e3","Type":"ContainerDied","Data":"295903563b9b067361e43f861a075e8e290d2485d4cdbc9531321c95f473281d"}
Sep 10 00:50:14.077440 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:14.077406    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltcj"
Sep 10 00:50:14.077580 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:14.077511    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8ltcj" podUID="fc958bde-e5aa-468a-ae10-0c49f9438168"
Sep 10 00:50:14.077580 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:14.077549    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l4w6p"
Sep 10 00:50:14.077661 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:14.077597    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l4w6p" podUID="00cec203-bdcf-4e75-acfa-29e1b784840e"
Sep 10 00:50:14.077661 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:14.077623    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cvjrx"
Sep 10 00:50:14.077715 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:14.077674    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cvjrx" podUID="50d35dde-26ab-4a1a-8f6a-2d663df04bc5"
Sep 10 00:50:15.265721 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:15.265533    2563 generic.go:356] "Generic (PLEG): container finished" podID="9f66caff-9501-4b7e-8986-9161a3ae70e3" containerID="3b46bdbe5a7036c9a07af58664d4f3af6e56118739b005da96c6934d58c83c69" exitCode=0
Sep 10 00:50:15.265721 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:15.265615    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6nzvr" event={"ID":"9f66caff-9501-4b7e-8986-9161a3ae70e3","Type":"ContainerDied","Data":"3b46bdbe5a7036c9a07af58664d4f3af6e56118739b005da96c6934d58c83c69"}
Sep 10 00:50:16.076529 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:16.076496    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltcj"
Sep 10 00:50:16.076678 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:16.076553    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cvjrx"
Sep 10 00:50:16.076678 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:16.076552    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l4w6p"
Sep 10 00:50:16.076678 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:16.076648    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8ltcj" podUID="fc958bde-e5aa-468a-ae10-0c49f9438168"
Sep 10 00:50:16.076837 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:16.076782    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cvjrx" podUID="50d35dde-26ab-4a1a-8f6a-2d663df04bc5"
Sep 10 00:50:16.076869 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:16.076848    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l4w6p" podUID="00cec203-bdcf-4e75-acfa-29e1b784840e"
Sep 10 00:50:16.270775 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:16.270736    2563 generic.go:356] "Generic (PLEG): container finished" podID="9f66caff-9501-4b7e-8986-9161a3ae70e3" containerID="f22a247bf12ed9d7596d0d94648a5f6f11dcea49590de2d283b80cf2188b69d2" exitCode=0
Sep 10 00:50:16.271163 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:16.270787    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6nzvr" event={"ID":"9f66caff-9501-4b7e-8986-9161a3ae70e3","Type":"ContainerDied","Data":"f22a247bf12ed9d7596d0d94648a5f6f11dcea49590de2d283b80cf2188b69d2"}
Sep 10 00:50:17.692562 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:17.692525    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/00cec203-bdcf-4e75-acfa-29e1b784840e-original-pull-secret\") pod \"global-pull-secret-syncer-l4w6p\" (UID: \"00cec203-bdcf-4e75-acfa-29e1b784840e\") " pod="kube-system/global-pull-secret-syncer-l4w6p"
Sep 10 00:50:17.693165 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:17.692760    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 00:50:17.693165 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:17.692878    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00cec203-bdcf-4e75-acfa-29e1b784840e-original-pull-secret podName:00cec203-bdcf-4e75-acfa-29e1b784840e nodeName:}" failed. No retries permitted until 2026-09-10 00:50:49.692859676 +0000 UTC m=+66.190159255 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/00cec203-bdcf-4e75-acfa-29e1b784840e-original-pull-secret") pod "global-pull-secret-syncer-l4w6p" (UID: "00cec203-bdcf-4e75-acfa-29e1b784840e") : object "kube-system"/"original-pull-secret" not registered
Sep 10 00:50:17.792897 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:17.792861    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc958bde-e5aa-468a-ae10-0c49f9438168-metrics-certs\") pod \"network-metrics-daemon-8ltcj\" (UID: \"fc958bde-e5aa-468a-ae10-0c49f9438168\") " pod="openshift-multus/network-metrics-daemon-8ltcj"
Sep 10 00:50:17.793072 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:17.793023    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 00:50:17.793121 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:17.793086    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc958bde-e5aa-468a-ae10-0c49f9438168-metrics-certs podName:fc958bde-e5aa-468a-ae10-0c49f9438168 nodeName:}" failed. No retries permitted until 2026-09-10 00:50:49.793071839 +0000 UTC m=+66.290371401 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fc958bde-e5aa-468a-ae10-0c49f9438168-metrics-certs") pod "network-metrics-daemon-8ltcj" (UID: "fc958bde-e5aa-468a-ae10-0c49f9438168") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 00:50:17.893950 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:17.893916    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kctp\" (UniqueName: \"kubernetes.io/projected/50d35dde-26ab-4a1a-8f6a-2d663df04bc5-kube-api-access-2kctp\") pod \"network-check-target-cvjrx\" (UID: \"50d35dde-26ab-4a1a-8f6a-2d663df04bc5\") " pod="openshift-network-diagnostics/network-check-target-cvjrx"
Sep 10 00:50:17.894126 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:17.894086    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 00:50:17.894126 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:17.894106    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 00:50:17.894126 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:17.894118    2563 projected.go:196] Error preparing data for projected volume kube-api-access-2kctp for pod openshift-network-diagnostics/network-check-target-cvjrx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 00:50:17.894287 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:17.894191    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50d35dde-26ab-4a1a-8f6a-2d663df04bc5-kube-api-access-2kctp podName:50d35dde-26ab-4a1a-8f6a-2d663df04bc5 nodeName:}" failed. No retries permitted until 2026-09-10 00:50:49.894163824 +0000 UTC m=+66.391463388 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-2kctp" (UniqueName: "kubernetes.io/projected/50d35dde-26ab-4a1a-8f6a-2d663df04bc5-kube-api-access-2kctp") pod "network-check-target-cvjrx" (UID: "50d35dde-26ab-4a1a-8f6a-2d663df04bc5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 00:50:18.076413 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.076339    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltcj"
Sep 10 00:50:18.076413 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.076366    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cvjrx"
Sep 10 00:50:18.076711 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.076337    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l4w6p"
Sep 10 00:50:18.079526 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:18.077042    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8ltcj" podUID="fc958bde-e5aa-468a-ae10-0c49f9438168"
Sep 10 00:50:18.079526 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:18.077182    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cvjrx" podUID="50d35dde-26ab-4a1a-8f6a-2d663df04bc5"
Sep 10 00:50:18.079526 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:18.077297    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l4w6p" podUID="00cec203-bdcf-4e75-acfa-29e1b784840e"
Sep 10 00:50:18.327875 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.327801    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-204.ec2.internal" event="NodeReady"
Sep 10 00:50:18.328045 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.327959    2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 10 00:50:18.365048 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.365015    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-b54b88b4b-q42sn"]
Sep 10 00:50:18.368690 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.368670    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b54b88b4b-q42sn"
Sep 10 00:50:18.372038 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.372007    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 10 00:50:18.372167 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.372099    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tmd2k\""
Sep 10 00:50:18.372167 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.372099    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 10 00:50:18.372575 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.372555    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 10 00:50:18.378828 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.378810    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 10 00:50:18.387425 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.387185    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b54b88b4b-q42sn"]
Sep 10 00:50:18.498773 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.498735    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-image-registry-private-configuration\") pod \"image-registry-b54b88b4b-q42sn\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") " pod="openshift-image-registry/image-registry-b54b88b4b-q42sn"
Sep 10 00:50:18.498939 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.498783    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-bound-sa-token\") pod \"image-registry-b54b88b4b-q42sn\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") " pod="openshift-image-registry/image-registry-b54b88b4b-q42sn"
Sep 10 00:50:18.498939 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.498810    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-trusted-ca\") pod \"image-registry-b54b88b4b-q42sn\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") " pod="openshift-image-registry/image-registry-b54b88b4b-q42sn"
Sep 10 00:50:18.498939 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.498827    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-ca-trust-extracted\") pod \"image-registry-b54b88b4b-q42sn\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") " pod="openshift-image-registry/image-registry-b54b88b4b-q42sn"
Sep 10 00:50:18.498939 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.498844    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd5tw\" (UniqueName: \"kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-kube-api-access-dd5tw\") pod \"image-registry-b54b88b4b-q42sn\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") " pod="openshift-image-registry/image-registry-b54b88b4b-q42sn"
Sep 10 00:50:18.498939 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.498898    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-registry-tls\") pod \"image-registry-b54b88b4b-q42sn\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") " pod="openshift-image-registry/image-registry-b54b88b4b-q42sn"
Sep 10 00:50:18.498939 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.498930    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-registry-certificates\") pod \"image-registry-b54b88b4b-q42sn\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") " pod="openshift-image-registry/image-registry-b54b88b4b-q42sn"
Sep 10 00:50:18.499187 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.498961    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-installation-pull-secrets\") pod \"image-registry-b54b88b4b-q42sn\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") " pod="openshift-image-registry/image-registry-b54b88b4b-q42sn"
Sep 10 00:50:18.537329 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.537295    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-kw2lh"]
Sep 10 00:50:18.541627 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.541602    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kw2lh"
Sep 10 00:50:18.544066 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.544046    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 10 00:50:18.544177 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.544133    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 10 00:50:18.544177 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.544140    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fhnl9\""
Sep 10 00:50:18.549198 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.549179    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kw2lh"]
Sep 10 00:50:18.575132 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.575099    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-sgpwp"]
Sep 10 00:50:18.579873 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.579796    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sgpwp"
Sep 10 00:50:18.582867 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.582622    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 10 00:50:18.582867 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.582673    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 10 00:50:18.582867 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.582676    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-j4fzs\""
Sep 10 00:50:18.582867 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.582719    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 10 00:50:18.588147 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.588125    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sgpwp"]
Sep 10 00:50:18.599960 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.599931    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-image-registry-private-configuration\") pod \"image-registry-b54b88b4b-q42sn\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") " pod="openshift-image-registry/image-registry-b54b88b4b-q42sn"
Sep 10 00:50:18.600101 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.599970    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-bound-sa-token\") pod \"image-registry-b54b88b4b-q42sn\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") " pod="openshift-image-registry/image-registry-b54b88b4b-q42sn"
Sep 10 00:50:18.600101 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.600022    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-trusted-ca\") pod \"image-registry-b54b88b4b-q42sn\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") " pod="openshift-image-registry/image-registry-b54b88b4b-q42sn"
Sep 10 00:50:18.600101 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.600052    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-ca-trust-extracted\") pod \"image-registry-b54b88b4b-q42sn\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") " pod="openshift-image-registry/image-registry-b54b88b4b-q42sn"
Sep 10 00:50:18.600101 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.600080    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd5tw\" (UniqueName: \"kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-kube-api-access-dd5tw\") pod \"image-registry-b54b88b4b-q42sn\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") " pod="openshift-image-registry/image-registry-b54b88b4b-q42sn"
Sep 10 00:50:18.600306 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.600121    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-registry-tls\") pod \"image-registry-b54b88b4b-q42sn\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") " pod="openshift-image-registry/image-registry-b54b88b4b-q42sn"
Sep 10 00:50:18.600306 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.600146    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-registry-certificates\") pod \"image-registry-b54b88b4b-q42sn\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") " pod="openshift-image-registry/image-registry-b54b88b4b-q42sn"
Sep 10 00:50:18.600306 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.600170    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-installation-pull-secrets\") pod \"image-registry-b54b88b4b-q42sn\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") " pod="openshift-image-registry/image-registry-b54b88b4b-q42sn"
Sep 10 00:50:18.600453 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:18.600407    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 00:50:18.600453 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:18.600428    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b54b88b4b-q42sn: secret "image-registry-tls" not found
Sep 10 00:50:18.600546 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:18.600491    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-registry-tls podName:8cf5ecf8-4fd5-4c6e-99ea-4015ff995450 nodeName:}" failed. No retries permitted until 2026-09-10 00:50:19.100470789 +0000 UTC m=+35.597770370 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-registry-tls") pod "image-registry-b54b88b4b-q42sn" (UID: "8cf5ecf8-4fd5-4c6e-99ea-4015ff995450") : secret "image-registry-tls" not found
Sep 10 00:50:18.600930 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.600638    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-ca-trust-extracted\") pod \"image-registry-b54b88b4b-q42sn\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") " pod="openshift-image-registry/image-registry-b54b88b4b-q42sn"
Sep 10 00:50:18.601263 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.601240    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-registry-certificates\") pod \"image-registry-b54b88b4b-q42sn\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") " pod="openshift-image-registry/image-registry-b54b88b4b-q42sn"
Sep 10 00:50:18.601678 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.601658    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-trusted-ca\") pod \"image-registry-b54b88b4b-q42sn\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") " pod="openshift-image-registry/image-registry-b54b88b4b-q42sn"
Sep 10 00:50:18.604553 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.604534    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-image-registry-private-configuration\") pod \"image-registry-b54b88b4b-q42sn\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") " pod="openshift-image-registry/image-registry-b54b88b4b-q42sn"
Sep 10 00:50:18.604653 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.604625    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-installation-pull-secrets\") pod \"image-registry-b54b88b4b-q42sn\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") " pod="openshift-image-registry/image-registry-b54b88b4b-q42sn"
Sep 10 00:50:18.613921 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.613896    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-bound-sa-token\") pod \"image-registry-b54b88b4b-q42sn\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") " pod="openshift-image-registry/image-registry-b54b88b4b-q42sn"
Sep 10 00:50:18.614336 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.614311    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd5tw\" (UniqueName: \"kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-kube-api-access-dd5tw\") pod \"image-registry-b54b88b4b-q42sn\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") " pod="openshift-image-registry/image-registry-b54b88b4b-q42sn"
Sep 10 00:50:18.701320 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.701286    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/58138b99-90e8-4b0a-89bf-891604fbaca8-metrics-tls\") pod \"dns-default-kw2lh\" (UID: \"58138b99-90e8-4b0a-89bf-891604fbaca8\") " pod="openshift-dns/dns-default-kw2lh"
Sep 10 00:50:18.701320 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.701318    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlxkm\" (UniqueName: \"kubernetes.io/projected/58138b99-90e8-4b0a-89bf-891604fbaca8-kube-api-access-dlxkm\") pod \"dns-default-kw2lh\" (UID: \"58138b99-90e8-4b0a-89bf-891604fbaca8\") " pod="openshift-dns/dns-default-kw2lh"
Sep 10 00:50:18.701718 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.701336    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/58138b99-90e8-4b0a-89bf-891604fbaca8-config-volume\") pod \"dns-default-kw2lh\" (UID: \"58138b99-90e8-4b0a-89bf-891604fbaca8\") " pod="openshift-dns/dns-default-kw2lh"
Sep 10 00:50:18.701718 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.701350    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58138b99-90e8-4b0a-89bf-891604fbaca8-tmp-dir\") pod \"dns-default-kw2lh\" (UID: \"58138b99-90e8-4b0a-89bf-891604fbaca8\") " pod="openshift-dns/dns-default-kw2lh"
Sep 10 00:50:18.701718 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.701383    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96504ff2-84b9-442b-a77e-254cbe9d8e5a-cert\") pod \"ingress-canary-sgpwp\" (UID: \"96504ff2-84b9-442b-a77e-254cbe9d8e5a\") " pod="openshift-ingress-canary/ingress-canary-sgpwp"
Sep 10 00:50:18.701718 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.701402    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2g4c\" (UniqueName: \"kubernetes.io/projected/96504ff2-84b9-442b-a77e-254cbe9d8e5a-kube-api-access-d2g4c\") pod \"ingress-canary-sgpwp\" (UID: \"96504ff2-84b9-442b-a77e-254cbe9d8e5a\") " pod="openshift-ingress-canary/ingress-canary-sgpwp"
Sep 10 00:50:18.802297 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.802258    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/58138b99-90e8-4b0a-89bf-891604fbaca8-metrics-tls\") pod \"dns-default-kw2lh\" (UID: \"58138b99-90e8-4b0a-89bf-891604fbaca8\") " pod="openshift-dns/dns-default-kw2lh"
Sep 10 00:50:18.802297 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.802303    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlxkm\" (UniqueName: \"kubernetes.io/projected/58138b99-90e8-4b0a-89bf-891604fbaca8-kube-api-access-dlxkm\") pod \"dns-default-kw2lh\" (UID: \"58138b99-90e8-4b0a-89bf-891604fbaca8\") " pod="openshift-dns/dns-default-kw2lh"
Sep 10 00:50:18.802540 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.802328    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/58138b99-90e8-4b0a-89bf-891604fbaca8-config-volume\") pod \"dns-default-kw2lh\" (UID: \"58138b99-90e8-4b0a-89bf-891604fbaca8\") " pod="openshift-dns/dns-default-kw2lh"
Sep 10 00:50:18.803313 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.803279    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58138b99-90e8-4b0a-89bf-891604fbaca8-tmp-dir\") pod \"dns-default-kw2lh\" (UID: \"58138b99-90e8-4b0a-89bf-891604fbaca8\") " pod="openshift-dns/dns-default-kw2lh"
Sep 10 00:50:18.803446 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:18.803308    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 00:50:18.803446 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.803353    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96504ff2-84b9-442b-a77e-254cbe9d8e5a-cert\") pod \"ingress-canary-sgpwp\" (UID: \"96504ff2-84b9-442b-a77e-254cbe9d8e5a\") " pod="openshift-ingress-canary/ingress-canary-sgpwp"
Sep 10 00:50:18.803446 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.803394    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2g4c\" (UniqueName: \"kubernetes.io/projected/96504ff2-84b9-442b-a77e-254cbe9d8e5a-kube-api-access-d2g4c\") pod \"ingress-canary-sgpwp\" (UID: \"96504ff2-84b9-442b-a77e-254cbe9d8e5a\") " pod="openshift-ingress-canary/ingress-canary-sgpwp"
Sep 10 00:50:18.803446 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:18.803410    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58138b99-90e8-4b0a-89bf-891604fbaca8-metrics-tls podName:58138b99-90e8-4b0a-89bf-891604fbaca8 nodeName:}" failed. No retries permitted until 2026-09-10 00:50:19.303388042 +0000 UTC m=+35.800687615 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/58138b99-90e8-4b0a-89bf-891604fbaca8-metrics-tls") pod "dns-default-kw2lh" (UID: "58138b99-90e8-4b0a-89bf-891604fbaca8") : secret "dns-default-metrics-tls" not found
Sep 10 00:50:18.806357 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.803828    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/58138b99-90e8-4b0a-89bf-891604fbaca8-config-volume\") pod \"dns-default-kw2lh\" (UID: \"58138b99-90e8-4b0a-89bf-891604fbaca8\") " pod="openshift-dns/dns-default-kw2lh"
Sep 10 00:50:18.806357 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.804029    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58138b99-90e8-4b0a-89bf-891604fbaca8-tmp-dir\") pod \"dns-default-kw2lh\" (UID: \"58138b99-90e8-4b0a-89bf-891604fbaca8\") " pod="openshift-dns/dns-default-kw2lh"
Sep 10 00:50:18.807189 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.807149    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96504ff2-84b9-442b-a77e-254cbe9d8e5a-cert\") pod \"ingress-canary-sgpwp\" (UID: \"96504ff2-84b9-442b-a77e-254cbe9d8e5a\") " pod="openshift-ingress-canary/ingress-canary-sgpwp"
Sep 10 00:50:18.813963 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.813942    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlxkm\" (UniqueName: \"kubernetes.io/projected/58138b99-90e8-4b0a-89bf-891604fbaca8-kube-api-access-dlxkm\") pod \"dns-default-kw2lh\" (UID: \"58138b99-90e8-4b0a-89bf-891604fbaca8\") " pod="openshift-dns/dns-default-kw2lh"
Sep 10 00:50:18.814248 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.814230    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2g4c\" (UniqueName: \"kubernetes.io/projected/96504ff2-84b9-442b-a77e-254cbe9d8e5a-kube-api-access-d2g4c\") pod \"ingress-canary-sgpwp\" (UID: \"96504ff2-84b9-442b-a77e-254cbe9d8e5a\") " pod="openshift-ingress-canary/ingress-canary-sgpwp"
Sep 10 00:50:18.890581 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:18.890492    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sgpwp"
Sep 10 00:50:19.106050 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:19.106014    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-registry-tls\") pod \"image-registry-b54b88b4b-q42sn\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") " pod="openshift-image-registry/image-registry-b54b88b4b-q42sn"
Sep 10 00:50:19.106254 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:19.106179    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 00:50:19.106254 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:19.106198    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b54b88b4b-q42sn: secret "image-registry-tls" not found
Sep 10 00:50:19.106369 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:19.106260    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-registry-tls podName:8cf5ecf8-4fd5-4c6e-99ea-4015ff995450 nodeName:}" failed. No retries permitted until 2026-09-10 00:50:20.10624239 +0000 UTC m=+36.603541947 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-registry-tls") pod "image-registry-b54b88b4b-q42sn" (UID: "8cf5ecf8-4fd5-4c6e-99ea-4015ff995450") : secret "image-registry-tls" not found
Sep 10 00:50:19.199737 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:19.199656    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sgpwp"]
Sep 10 00:50:19.203891 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:50:19.203867    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96504ff2_84b9_442b_a77e_254cbe9d8e5a.slice/crio-17c9b5d792141c2bc0bea20ed2a0b990d4c01a687ce245d6b52f8dbde74f9dc3 WatchSource:0}: Error finding container 17c9b5d792141c2bc0bea20ed2a0b990d4c01a687ce245d6b52f8dbde74f9dc3: Status 404 returned error can't find the container with id 17c9b5d792141c2bc0bea20ed2a0b990d4c01a687ce245d6b52f8dbde74f9dc3
Sep 10 00:50:19.278732 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:19.278501    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sgpwp" event={"ID":"96504ff2-84b9-442b-a77e-254cbe9d8e5a","Type":"ContainerStarted","Data":"17c9b5d792141c2bc0bea20ed2a0b990d4c01a687ce245d6b52f8dbde74f9dc3"}
Sep 10 00:50:19.308675 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:19.308632    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/58138b99-90e8-4b0a-89bf-891604fbaca8-metrics-tls\") pod \"dns-default-kw2lh\" (UID: \"58138b99-90e8-4b0a-89bf-891604fbaca8\") " pod="openshift-dns/dns-default-kw2lh"
Sep 10 00:50:19.308855 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:19.308754    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 00:50:19.308855 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:19.308821    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58138b99-90e8-4b0a-89bf-891604fbaca8-metrics-tls podName:58138b99-90e8-4b0a-89bf-891604fbaca8 nodeName:}" failed. No retries permitted until 2026-09-10 00:50:20.308801942 +0000 UTC m=+36.806101524 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/58138b99-90e8-4b0a-89bf-891604fbaca8-metrics-tls") pod "dns-default-kw2lh" (UID: "58138b99-90e8-4b0a-89bf-891604fbaca8") : secret "dns-default-metrics-tls" not found
Sep 10 00:50:20.077032 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:20.076977    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l4w6p"
Sep 10 00:50:20.077032 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:20.077022    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltcj"
Sep 10 00:50:20.077547 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:20.076997    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cvjrx"
Sep 10 00:50:20.079792 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:20.079759    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 00:50:20.079792 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:20.079773    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 00:50:20.081364 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:20.081127    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 00:50:20.081364 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:20.081156    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mhb4c\""
Sep 10 00:50:20.081364 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:20.081188    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rcm88\""
Sep 10 00:50:20.081601 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:20.081416    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 00:50:20.114684 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:20.114656    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-registry-tls\") pod \"image-registry-b54b88b4b-q42sn\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") " pod="openshift-image-registry/image-registry-b54b88b4b-q42sn"
Sep 10 00:50:20.114925 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:20.114905    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 00:50:20.115026 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:20.114929    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b54b88b4b-q42sn: secret "image-registry-tls" not found
Sep 10 00:50:20.115026 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:20.115006    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-registry-tls podName:8cf5ecf8-4fd5-4c6e-99ea-4015ff995450 nodeName:}" failed. No retries permitted until 2026-09-10 00:50:22.114971928 +0000 UTC m=+38.612271490 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-registry-tls") pod "image-registry-b54b88b4b-q42sn" (UID: "8cf5ecf8-4fd5-4c6e-99ea-4015ff995450") : secret "image-registry-tls" not found
Sep 10 00:50:20.293937 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:20.293896    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qhbzc_caff9962-15c8-4c05-ad49-a748c94a3b85/konnectivity-agent/0.log" limit=1024
Sep 10 00:50:20.316775 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:20.316736    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/58138b99-90e8-4b0a-89bf-891604fbaca8-metrics-tls\") pod \"dns-default-kw2lh\" (UID: \"58138b99-90e8-4b0a-89bf-891604fbaca8\") " pod="openshift-dns/dns-default-kw2lh"
Sep 10 00:50:20.316964 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:20.316923    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 00:50:20.317081 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:20.317023    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58138b99-90e8-4b0a-89bf-891604fbaca8-metrics-tls podName:58138b99-90e8-4b0a-89bf-891604fbaca8 nodeName:}" failed. No retries permitted until 2026-09-10 00:50:22.317002109 +0000 UTC m=+38.814301677 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/58138b99-90e8-4b0a-89bf-891604fbaca8-metrics-tls") pod "dns-default-kw2lh" (UID: "58138b99-90e8-4b0a-89bf-891604fbaca8") : secret "dns-default-metrics-tls" not found
Sep 10 00:50:20.405752 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:20.405655    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qhbzc_caff9962-15c8-4c05-ad49-a748c94a3b85/konnectivity-agent/0.log" limit=1024
Sep 10 00:50:20.502309 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:20.502271    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qhbzc_caff9962-15c8-4c05-ad49-a748c94a3b85/konnectivity-agent/0.log" limit=1024
Sep 10 00:50:20.605254 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:20.605219    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qhbzc_caff9962-15c8-4c05-ad49-a748c94a3b85/konnectivity-agent/0.log" limit=1024
Sep 10 00:50:20.847804 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:20.847719    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qhbzc_caff9962-15c8-4c05-ad49-a748c94a3b85/konnectivity-agent/0.log" limit=1024
Sep 10 00:50:20.969227 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:20.969180    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qhbzc_caff9962-15c8-4c05-ad49-a748c94a3b85/konnectivity-agent/0.log" limit=1024
Sep 10 00:50:21.645448 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:21.645412    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qhbzc_caff9962-15c8-4c05-ad49-a748c94a3b85/konnectivity-agent/0.log" limit=1024
Sep 10 00:50:21.705358 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:21.705325    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qhbzc_caff9962-15c8-4c05-ad49-a748c94a3b85/konnectivity-agent/0.log" limit=1024
Sep 10 00:50:22.042199 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:22.042092    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qhbzc_caff9962-15c8-4c05-ad49-a748c94a3b85/konnectivity-agent/0.log" limit=1024
Sep 10 00:50:22.094783 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:22.094733    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qhbzc_caff9962-15c8-4c05-ad49-a748c94a3b85/konnectivity-agent/0.log" limit=1024
Sep 10 00:50:22.133496 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:22.133458    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-registry-tls\") pod \"image-registry-b54b88b4b-q42sn\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") " pod="openshift-image-registry/image-registry-b54b88b4b-q42sn"
Sep 10 00:50:22.133681 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:22.133623    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 00:50:22.133681 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:22.133645    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b54b88b4b-q42sn: secret "image-registry-tls" not found
Sep 10 00:50:22.133797 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:22.133699    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-registry-tls podName:8cf5ecf8-4fd5-4c6e-99ea-4015ff995450 nodeName:}" failed. No retries permitted until 2026-09-10 00:50:26.133682979 +0000 UTC m=+42.630982536 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-registry-tls") pod "image-registry-b54b88b4b-q42sn" (UID: "8cf5ecf8-4fd5-4c6e-99ea-4015ff995450") : secret "image-registry-tls" not found
Sep 10 00:50:22.335492 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:22.335413    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/58138b99-90e8-4b0a-89bf-891604fbaca8-metrics-tls\") pod \"dns-default-kw2lh\" (UID: \"58138b99-90e8-4b0a-89bf-891604fbaca8\") " pod="openshift-dns/dns-default-kw2lh"
Sep 10 00:50:22.335645 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:22.335553    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 00:50:22.335645 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:22.335613    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58138b99-90e8-4b0a-89bf-891604fbaca8-metrics-tls podName:58138b99-90e8-4b0a-89bf-891604fbaca8 nodeName:}" failed. No retries permitted until 2026-09-10 00:50:26.335596933 +0000 UTC m=+42.832896511 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/58138b99-90e8-4b0a-89bf-891604fbaca8-metrics-tls") pod "dns-default-kw2lh" (UID: "58138b99-90e8-4b0a-89bf-891604fbaca8") : secret "dns-default-metrics-tls" not found
Sep 10 00:50:23.290106 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:23.290023    2563 generic.go:356] "Generic (PLEG): container finished" podID="9f66caff-9501-4b7e-8986-9161a3ae70e3" containerID="0dcab488657f86a8762bd58a70613aa8c6c1f99ca0a9268c09cbfeb9dd49c4be" exitCode=0
Sep 10 00:50:23.290946 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:23.290107    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6nzvr" event={"ID":"9f66caff-9501-4b7e-8986-9161a3ae70e3","Type":"ContainerDied","Data":"0dcab488657f86a8762bd58a70613aa8c6c1f99ca0a9268c09cbfeb9dd49c4be"}
Sep 10 00:50:23.291580 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:23.291551    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sgpwp" event={"ID":"96504ff2-84b9-442b-a77e-254cbe9d8e5a","Type":"ContainerStarted","Data":"87d8f0e46e36cdc0d46500a827c41e9cb4981fdfd66d12cca9d5cc17dcf32d14"}
Sep 10 00:50:23.326081 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:23.326039    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-sgpwp" podStartSLOduration=1.753164902 podStartE2EDuration="5.326025367s" totalImagesPullingTime="3.572860465s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:50:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:50:23.325525179 +0000 UTC m=+39.822824771" watchObservedRunningTime="2026-09-10 00:50:23.326025367 +0000 UTC m=+39.823324943"
Sep 10 00:50:23.772065 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:23.771900    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qhbzc_caff9962-15c8-4c05-ad49-a748c94a3b85/konnectivity-agent/0.log" limit=1024
Sep 10 00:50:23.872599 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:23.872560    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qhbzc_caff9962-15c8-4c05-ad49-a748c94a3b85/konnectivity-agent/0.log" limit=1024
Sep 10 00:50:24.296437 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:24.296404    2563 generic.go:356] "Generic (PLEG): container finished" podID="9f66caff-9501-4b7e-8986-9161a3ae70e3" containerID="3b9a1316745f0857abf3fba2098e6895505484f34999cd0806684ed56db596cf" exitCode=0
Sep 10 00:50:24.296815 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:24.296492    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6nzvr" event={"ID":"9f66caff-9501-4b7e-8986-9161a3ae70e3","Type":"ContainerDied","Data":"3b9a1316745f0857abf3fba2098e6895505484f34999cd0806684ed56db596cf"}
Sep 10 00:50:25.302082 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:25.302048    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6nzvr" event={"ID":"9f66caff-9501-4b7e-8986-9161a3ae70e3","Type":"ContainerStarted","Data":"1f6026be1f2f0dba7fc20cfaca4c8679836bb6e92054cbf5e34d4b918dd0c6b7"}
Sep 10 00:50:25.322591 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:25.322542    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6nzvr" podStartSLOduration=7.034895821 podStartE2EDuration="40.322527696s" totalImagesPullingTime="33.287631875s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 00:49:45 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:50:25.320862218 +0000 UTC m=+41.818161797" watchObservedRunningTime="2026-09-10 00:50:25.322527696 +0000 UTC m=+41.819827275"
Sep 10 00:50:25.440421 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:25.440387    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qhbzc_caff9962-15c8-4c05-ad49-a748c94a3b85/konnectivity-agent/0.log" limit=1024
Sep 10 00:50:25.493359 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:25.493329    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qhbzc_caff9962-15c8-4c05-ad49-a748c94a3b85/konnectivity-agent/0.log" limit=1024
Sep 10 00:50:26.163337 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:26.163290    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-registry-tls\") pod \"image-registry-b54b88b4b-q42sn\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") " pod="openshift-image-registry/image-registry-b54b88b4b-q42sn"
Sep 10 00:50:26.163520 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:26.163442    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 00:50:26.163520 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:26.163463    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b54b88b4b-q42sn: secret "image-registry-tls" not found
Sep 10 00:50:26.163520 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:26.163517    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-registry-tls podName:8cf5ecf8-4fd5-4c6e-99ea-4015ff995450 nodeName:}" failed. No retries permitted until 2026-09-10 00:50:34.163501975 +0000 UTC m=+50.660801537 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-registry-tls") pod "image-registry-b54b88b4b-q42sn" (UID: "8cf5ecf8-4fd5-4c6e-99ea-4015ff995450") : secret "image-registry-tls" not found
Sep 10 00:50:26.364625 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:26.364574    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/58138b99-90e8-4b0a-89bf-891604fbaca8-metrics-tls\") pod \"dns-default-kw2lh\" (UID: \"58138b99-90e8-4b0a-89bf-891604fbaca8\") " pod="openshift-dns/dns-default-kw2lh"
Sep 10 00:50:26.365040 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:26.364693    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 00:50:26.365040 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:26.364745    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58138b99-90e8-4b0a-89bf-891604fbaca8-metrics-tls podName:58138b99-90e8-4b0a-89bf-891604fbaca8 nodeName:}" failed. No retries permitted until 2026-09-10 00:50:34.364732349 +0000 UTC m=+50.862031905 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/58138b99-90e8-4b0a-89bf-891604fbaca8-metrics-tls") pod "dns-default-kw2lh" (UID: "58138b99-90e8-4b0a-89bf-891604fbaca8") : secret "dns-default-metrics-tls" not found
Sep 10 00:50:27.908332 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:27.908302    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qhbzc_caff9962-15c8-4c05-ad49-a748c94a3b85/konnectivity-agent/0.log" limit=1024
Sep 10 00:50:27.962401 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:27.962365    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qhbzc_caff9962-15c8-4c05-ad49-a748c94a3b85/konnectivity-agent/0.log" limit=1024
Sep 10 00:50:34.218125 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:34.218083    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-registry-tls\") pod \"image-registry-b54b88b4b-q42sn\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") " pod="openshift-image-registry/image-registry-b54b88b4b-q42sn"
Sep 10 00:50:34.218576 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:34.218243    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 00:50:34.218576 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:34.218259    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b54b88b4b-q42sn: secret "image-registry-tls" not found
Sep 10 00:50:34.218576 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:34.218321    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-registry-tls podName:8cf5ecf8-4fd5-4c6e-99ea-4015ff995450 nodeName:}" failed. No retries permitted until 2026-09-10 00:50:50.21830336 +0000 UTC m=+66.715602928 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-registry-tls") pod "image-registry-b54b88b4b-q42sn" (UID: "8cf5ecf8-4fd5-4c6e-99ea-4015ff995450") : secret "image-registry-tls" not found
Sep 10 00:50:34.420219 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:34.420180    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/58138b99-90e8-4b0a-89bf-891604fbaca8-metrics-tls\") pod \"dns-default-kw2lh\" (UID: \"58138b99-90e8-4b0a-89bf-891604fbaca8\") " pod="openshift-dns/dns-default-kw2lh"
Sep 10 00:50:34.420441 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:34.420293    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 00:50:34.420441 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:34.420343    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58138b99-90e8-4b0a-89bf-891604fbaca8-metrics-tls podName:58138b99-90e8-4b0a-89bf-891604fbaca8 nodeName:}" failed. No retries permitted until 2026-09-10 00:50:50.420330046 +0000 UTC m=+66.917629602 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/58138b99-90e8-4b0a-89bf-891604fbaca8-metrics-tls") pod "dns-default-kw2lh" (UID: "58138b99-90e8-4b0a-89bf-891604fbaca8") : secret "dns-default-metrics-tls" not found
Sep 10 00:50:40.118471 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:40.118433    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qhbzc_caff9962-15c8-4c05-ad49-a748c94a3b85/konnectivity-agent/0.log" limit=1024
Sep 10 00:50:40.176998 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:40.176951    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qhbzc_caff9962-15c8-4c05-ad49-a748c94a3b85/konnectivity-agent/0.log" limit=1024
Sep 10 00:50:40.491973 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:40.491895    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qhbzc_caff9962-15c8-4c05-ad49-a748c94a3b85/konnectivity-agent/0.log" limit=1024
Sep 10 00:50:40.591929 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:40.591898    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qhbzc_caff9962-15c8-4c05-ad49-a748c94a3b85/konnectivity-agent/0.log" limit=1024
Sep 10 00:50:40.902875 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:40.902840    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qhbzc_caff9962-15c8-4c05-ad49-a748c94a3b85/konnectivity-agent/0.log" limit=1024
Sep 10 00:50:41.270733 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:41.270647    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qhbzc_caff9962-15c8-4c05-ad49-a748c94a3b85/konnectivity-agent/0.log" limit=1024
Sep 10 00:50:41.328306 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:41.328277    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qhbzc_caff9962-15c8-4c05-ad49-a748c94a3b85/konnectivity-agent/0.log" limit=1024
Sep 10 00:50:44.278834 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:44.278805    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8nwz8"
Sep 10 00:50:49.723368 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:49.723306    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/00cec203-bdcf-4e75-acfa-29e1b784840e-original-pull-secret\") pod \"global-pull-secret-syncer-l4w6p\" (UID: \"00cec203-bdcf-4e75-acfa-29e1b784840e\") " pod="kube-system/global-pull-secret-syncer-l4w6p"
Sep 10 00:50:49.726237 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:49.726214    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 00:50:49.736752 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:49.736724    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/00cec203-bdcf-4e75-acfa-29e1b784840e-original-pull-secret\") pod \"global-pull-secret-syncer-l4w6p\" (UID: \"00cec203-bdcf-4e75-acfa-29e1b784840e\") " pod="kube-system/global-pull-secret-syncer-l4w6p"
Sep 10 00:50:49.792027 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:49.791974    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l4w6p"
Sep 10 00:50:49.824217 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:49.824178    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc958bde-e5aa-468a-ae10-0c49f9438168-metrics-certs\") pod \"network-metrics-daemon-8ltcj\" (UID: \"fc958bde-e5aa-468a-ae10-0c49f9438168\") " pod="openshift-multus/network-metrics-daemon-8ltcj"
Sep 10 00:50:49.826830 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:49.826807    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 00:50:49.835079 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:49.835047    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 10 00:50:49.835215 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:49.835164    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc958bde-e5aa-468a-ae10-0c49f9438168-metrics-certs podName:fc958bde-e5aa-468a-ae10-0c49f9438168 nodeName:}" failed. No retries permitted until 2026-09-10 00:51:53.835139338 +0000 UTC m=+130.332438914 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fc958bde-e5aa-468a-ae10-0c49f9438168-metrics-certs") pod "network-metrics-daemon-8ltcj" (UID: "fc958bde-e5aa-468a-ae10-0c49f9438168") : secret "metrics-daemon-secret" not found
Sep 10 00:50:49.925138 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:49.925083    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kctp\" (UniqueName: \"kubernetes.io/projected/50d35dde-26ab-4a1a-8f6a-2d663df04bc5-kube-api-access-2kctp\") pod \"network-check-target-cvjrx\" (UID: \"50d35dde-26ab-4a1a-8f6a-2d663df04bc5\") " pod="openshift-network-diagnostics/network-check-target-cvjrx"
Sep 10 00:50:49.927910 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:49.927883    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 00:50:49.938223 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:49.938192    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 00:50:49.948453 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:49.948422    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kctp\" (UniqueName: \"kubernetes.io/projected/50d35dde-26ab-4a1a-8f6a-2d663df04bc5-kube-api-access-2kctp\") pod \"network-check-target-cvjrx\" (UID: \"50d35dde-26ab-4a1a-8f6a-2d663df04bc5\") " pod="openshift-network-diagnostics/network-check-target-cvjrx"
Sep 10 00:50:50.058758 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:50.058688    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-l4w6p"]
Sep 10 00:50:50.063385 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:50:50.063357    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00cec203_bdcf_4e75_acfa_29e1b784840e.slice/crio-1a1516a7fc22cb17dc728143aed7bd248c0c341efdffe0eaf23c623a32d041c4 WatchSource:0}: Error finding container 1a1516a7fc22cb17dc728143aed7bd248c0c341efdffe0eaf23c623a32d041c4: Status 404 returned error can't find the container with id 1a1516a7fc22cb17dc728143aed7bd248c0c341efdffe0eaf23c623a32d041c4
Sep 10 00:50:50.103161 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:50.103132    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rcm88\""
Sep 10 00:50:50.111294 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:50.111269    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cvjrx"
Sep 10 00:50:50.226893 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:50.226851    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-registry-tls\") pod \"image-registry-b54b88b4b-q42sn\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") " pod="openshift-image-registry/image-registry-b54b88b4b-q42sn"
Sep 10 00:50:50.227081 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:50.227024    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 00:50:50.227081 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:50.227046    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b54b88b4b-q42sn: secret "image-registry-tls" not found
Sep 10 00:50:50.227168 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:50.227103    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-registry-tls podName:8cf5ecf8-4fd5-4c6e-99ea-4015ff995450 nodeName:}" failed. No retries permitted until 2026-09-10 00:51:22.227086299 +0000 UTC m=+98.724385869 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-registry-tls") pod "image-registry-b54b88b4b-q42sn" (UID: "8cf5ecf8-4fd5-4c6e-99ea-4015ff995450") : secret "image-registry-tls" not found
Sep 10 00:50:50.361002 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:50.360951    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-l4w6p" event={"ID":"00cec203-bdcf-4e75-acfa-29e1b784840e","Type":"ContainerStarted","Data":"1a1516a7fc22cb17dc728143aed7bd248c0c341efdffe0eaf23c623a32d041c4"}
Sep 10 00:50:50.377910 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:50.377879    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cvjrx"]
Sep 10 00:50:50.380923 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:50:50.380901    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50d35dde_26ab_4a1a_8f6a_2d663df04bc5.slice/crio-af26516103bb8eab5dcc81a32ca79411957438cb88798d47d2cfae1b291b7238 WatchSource:0}: Error finding container af26516103bb8eab5dcc81a32ca79411957438cb88798d47d2cfae1b291b7238: Status 404 returned error can't find the container with id af26516103bb8eab5dcc81a32ca79411957438cb88798d47d2cfae1b291b7238
Sep 10 00:50:50.428362 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:50.428321    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/58138b99-90e8-4b0a-89bf-891604fbaca8-metrics-tls\") pod \"dns-default-kw2lh\" (UID: \"58138b99-90e8-4b0a-89bf-891604fbaca8\") " pod="openshift-dns/dns-default-kw2lh"
Sep 10 00:50:50.428533 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:50.428435    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 00:50:50.428533 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:50.428485    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58138b99-90e8-4b0a-89bf-891604fbaca8-metrics-tls podName:58138b99-90e8-4b0a-89bf-891604fbaca8 nodeName:}" failed. No retries permitted until 2026-09-10 00:51:22.428472953 +0000 UTC m=+98.925772510 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/58138b99-90e8-4b0a-89bf-891604fbaca8-metrics-tls") pod "dns-default-kw2lh" (UID: "58138b99-90e8-4b0a-89bf-891604fbaca8") : secret "dns-default-metrics-tls" not found
Sep 10 00:50:51.365767 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.365730    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cvjrx" event={"ID":"50d35dde-26ab-4a1a-8f6a-2d663df04bc5","Type":"ContainerStarted","Data":"af26516103bb8eab5dcc81a32ca79411957438cb88798d47d2cfae1b291b7238"}
Sep 10 00:50:51.805326 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.805237    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-gzrtx"]
Sep 10 00:50:51.811528 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.811475    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-gzrtx"
Sep 10 00:50:51.813269 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.813230    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-rpj5t"]
Sep 10 00:50:51.815712 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.815689    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-sbxs6\""
Sep 10 00:50:51.815927 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.815911    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 10 00:50:51.816194 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.816173    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 10 00:50:51.818116 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.817253    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-4qhvr"]
Sep 10 00:50:51.818116 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.818104    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rpj5t"
Sep 10 00:50:51.821139 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.821111    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 10 00:50:51.821234 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.821175    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 10 00:50:51.821380 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.821364    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 10 00:50:51.821654 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.821637    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-r5fkd\""
Sep 10 00:50:51.822312 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.821820    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-nnwxp"]
Sep 10 00:50:51.822312 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.821832    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 10 00:50:51.822312 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.822066    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-4qhvr"
Sep 10 00:50:51.826489 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.825700    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-kdm6r"]
Sep 10 00:50:51.826755 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.826733    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-6w9tn\""
Sep 10 00:50:51.826965 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.826949    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 10 00:50:51.827416 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.827391    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-nnwxp"
Sep 10 00:50:51.828291 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.828268    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 10 00:50:51.828431 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.828412    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 10 00:50:51.829211 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.828640    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 10 00:50:51.829211 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.828720    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-gzrtx"]
Sep 10 00:50:51.829211 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.828807    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-kdm6r"
Sep 10 00:50:51.830229 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.830211    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-rpj5t"]
Sep 10 00:50:51.830793 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.830771    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-4qhvr"]
Sep 10 00:50:51.830883 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.830812    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 10 00:50:51.831107 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.831050    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 10 00:50:51.831685 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.831285    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 10 00:50:51.831685 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.831575    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-jldjv\""
Sep 10 00:50:51.831872 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.831806    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 10 00:50:51.833459 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.833419    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 10 00:50:51.834120 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.834100    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-kdm6r"]
Sep 10 00:50:51.834262 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.834221    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 10 00:50:51.834806 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.834789    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 10 00:50:51.835070 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.834831    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-vbkkz\""
Sep 10 00:50:51.835404 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.835384    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 10 00:50:51.836915 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.836895    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-nnwxp"]
Sep 10 00:50:51.838580 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.838550    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1cee7771-6381-4fad-8c1e-ad1098fe409b-config\") pod \"console-operator-85b454874b-nnwxp\" (UID: \"1cee7771-6381-4fad-8c1e-ad1098fe409b\") " pod="openshift-console-operator/console-operator-85b454874b-nnwxp"
Sep 10 00:50:51.838677 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.838589    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8wrd\" (UniqueName: \"kubernetes.io/projected/784a9928-9ed1-4f75-afdb-c42ee2030777-kube-api-access-p8wrd\") pod \"cluster-monitoring-operator-6cdcc96965-4qhvr\" (UID: \"784a9928-9ed1-4f75-afdb-c42ee2030777\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-4qhvr"
Sep 10 00:50:51.838677 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.838618    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/501b4642-4933-4505-afe4-4446b75ce2ee-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-kdm6r\" (UID: \"501b4642-4933-4505-afe4-4446b75ce2ee\") " pod="openshift-insights/insights-operator-76bcd6ccf5-kdm6r"
Sep 10 00:50:51.838677 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.838662    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/784a9928-9ed1-4f75-afdb-c42ee2030777-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-4qhvr\" (UID: \"784a9928-9ed1-4f75-afdb-c42ee2030777\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-4qhvr"
Sep 10 00:50:51.838843 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.838695    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6ebec561-8cb9-40af-92f8-dae61b52833b-config\") pod \"service-ca-operator-859f94c4b-rpj5t\" (UID: \"6ebec561-8cb9-40af-92f8-dae61b52833b\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rpj5t"
Sep 10 00:50:51.838843 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.838718    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1cee7771-6381-4fad-8c1e-ad1098fe409b-trusted-ca\") pod \"console-operator-85b454874b-nnwxp\" (UID: \"1cee7771-6381-4fad-8c1e-ad1098fe409b\") " pod="openshift-console-operator/console-operator-85b454874b-nnwxp"
Sep 10 00:50:51.838843 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.838742    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/501b4642-4933-4505-afe4-4446b75ce2ee-serving-cert\") pod \"insights-operator-76bcd6ccf5-kdm6r\" (UID: \"501b4642-4933-4505-afe4-4446b75ce2ee\") " pod="openshift-insights/insights-operator-76bcd6ccf5-kdm6r"
Sep 10 00:50:51.838843 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.838765    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx5vm\" (UniqueName: \"kubernetes.io/projected/501b4642-4933-4505-afe4-4446b75ce2ee-kube-api-access-tx5vm\") pod \"insights-operator-76bcd6ccf5-kdm6r\" (UID: \"501b4642-4933-4505-afe4-4446b75ce2ee\") " pod="openshift-insights/insights-operator-76bcd6ccf5-kdm6r"
Sep 10 00:50:51.838843 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.838814    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6ebec561-8cb9-40af-92f8-dae61b52833b-serving-cert\") pod \"service-ca-operator-859f94c4b-rpj5t\" (UID: \"6ebec561-8cb9-40af-92f8-dae61b52833b\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rpj5t"
Sep 10 00:50:51.838843 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.838838    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/784a9928-9ed1-4f75-afdb-c42ee2030777-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-4qhvr\" (UID: \"784a9928-9ed1-4f75-afdb-c42ee2030777\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-4qhvr"
Sep 10 00:50:51.839158 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.838872    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b71eb054-e5f2-4f16-b9a3-1253c28778ca-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-gzrtx\" (UID: \"b71eb054-e5f2-4f16-b9a3-1253c28778ca\") " pod="openshift-network-console/networking-console-plugin-79977f9844-gzrtx"
Sep 10 00:50:51.839158 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.838903    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/501b4642-4933-4505-afe4-4446b75ce2ee-snapshots\") pod \"insights-operator-76bcd6ccf5-kdm6r\" (UID: \"501b4642-4933-4505-afe4-4446b75ce2ee\") " pod="openshift-insights/insights-operator-76bcd6ccf5-kdm6r"
Sep 10 00:50:51.839158 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.838932    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/501b4642-4933-4505-afe4-4446b75ce2ee-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-kdm6r\" (UID: \"501b4642-4933-4505-afe4-4446b75ce2ee\") " pod="openshift-insights/insights-operator-76bcd6ccf5-kdm6r"
Sep 10 00:50:51.839158 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.839081    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b71eb054-e5f2-4f16-b9a3-1253c28778ca-nginx-conf\") pod \"networking-console-plugin-79977f9844-gzrtx\" (UID: \"b71eb054-e5f2-4f16-b9a3-1253c28778ca\") " pod="openshift-network-console/networking-console-plugin-79977f9844-gzrtx"
Sep 10 00:50:51.839158 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.839114    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/501b4642-4933-4505-afe4-4446b75ce2ee-tmp\") pod \"insights-operator-76bcd6ccf5-kdm6r\" (UID: \"501b4642-4933-4505-afe4-4446b75ce2ee\") " pod="openshift-insights/insights-operator-76bcd6ccf5-kdm6r"
Sep 10 00:50:51.839398 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.839184    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1cee7771-6381-4fad-8c1e-ad1098fe409b-serving-cert\") pod \"console-operator-85b454874b-nnwxp\" (UID: \"1cee7771-6381-4fad-8c1e-ad1098fe409b\") " pod="openshift-console-operator/console-operator-85b454874b-nnwxp"
Sep 10 00:50:51.839398 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.839290    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hmsh\" (UniqueName: \"kubernetes.io/projected/6ebec561-8cb9-40af-92f8-dae61b52833b-kube-api-access-5hmsh\") pod \"service-ca-operator-859f94c4b-rpj5t\" (UID: \"6ebec561-8cb9-40af-92f8-dae61b52833b\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rpj5t"
Sep 10 00:50:51.839398 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.839322    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg485\" (UniqueName: \"kubernetes.io/projected/1cee7771-6381-4fad-8c1e-ad1098fe409b-kube-api-access-tg485\") pod \"console-operator-85b454874b-nnwxp\" (UID: \"1cee7771-6381-4fad-8c1e-ad1098fe409b\") " pod="openshift-console-operator/console-operator-85b454874b-nnwxp"
Sep 10 00:50:51.839552 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.839514    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 10 00:50:51.840230 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.840211    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 10 00:50:51.940714 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.940675    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b71eb054-e5f2-4f16-b9a3-1253c28778ca-nginx-conf\") pod \"networking-console-plugin-79977f9844-gzrtx\" (UID: \"b71eb054-e5f2-4f16-b9a3-1253c28778ca\") " pod="openshift-network-console/networking-console-plugin-79977f9844-gzrtx"
Sep 10 00:50:51.940909 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.940727    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/501b4642-4933-4505-afe4-4446b75ce2ee-tmp\") pod \"insights-operator-76bcd6ccf5-kdm6r\" (UID: \"501b4642-4933-4505-afe4-4446b75ce2ee\") " pod="openshift-insights/insights-operator-76bcd6ccf5-kdm6r"
Sep 10 00:50:51.940909 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.940760    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1cee7771-6381-4fad-8c1e-ad1098fe409b-serving-cert\") pod \"console-operator-85b454874b-nnwxp\" (UID: \"1cee7771-6381-4fad-8c1e-ad1098fe409b\") " pod="openshift-console-operator/console-operator-85b454874b-nnwxp"
Sep 10 00:50:51.941051 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.940917    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hmsh\" (UniqueName: \"kubernetes.io/projected/6ebec561-8cb9-40af-92f8-dae61b52833b-kube-api-access-5hmsh\") pod \"service-ca-operator-859f94c4b-rpj5t\" (UID: \"6ebec561-8cb9-40af-92f8-dae61b52833b\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rpj5t"
Sep 10 00:50:51.941051 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.940946    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tg485\" (UniqueName: \"kubernetes.io/projected/1cee7771-6381-4fad-8c1e-ad1098fe409b-kube-api-access-tg485\") pod \"console-operator-85b454874b-nnwxp\" (UID: \"1cee7771-6381-4fad-8c1e-ad1098fe409b\") " pod="openshift-console-operator/console-operator-85b454874b-nnwxp"
Sep 10 00:50:51.941051 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.940977    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1cee7771-6381-4fad-8c1e-ad1098fe409b-config\") pod \"console-operator-85b454874b-nnwxp\" (UID: \"1cee7771-6381-4fad-8c1e-ad1098fe409b\") " pod="openshift-console-operator/console-operator-85b454874b-nnwxp"
Sep 10 00:50:51.941051 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.941018    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8wrd\" (UniqueName: \"kubernetes.io/projected/784a9928-9ed1-4f75-afdb-c42ee2030777-kube-api-access-p8wrd\") pod \"cluster-monitoring-operator-6cdcc96965-4qhvr\" (UID: \"784a9928-9ed1-4f75-afdb-c42ee2030777\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-4qhvr"
Sep 10 00:50:51.941051 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.941043    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/501b4642-4933-4505-afe4-4446b75ce2ee-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-kdm6r\" (UID: \"501b4642-4933-4505-afe4-4446b75ce2ee\") " pod="openshift-insights/insights-operator-76bcd6ccf5-kdm6r"
Sep 10 00:50:51.941307 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.941081    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/784a9928-9ed1-4f75-afdb-c42ee2030777-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-4qhvr\" (UID: \"784a9928-9ed1-4f75-afdb-c42ee2030777\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-4qhvr"
Sep 10 00:50:51.941307 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.941109    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6ebec561-8cb9-40af-92f8-dae61b52833b-config\") pod \"service-ca-operator-859f94c4b-rpj5t\" (UID: \"6ebec561-8cb9-40af-92f8-dae61b52833b\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rpj5t"
Sep 10 00:50:51.941307 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.941130    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1cee7771-6381-4fad-8c1e-ad1098fe409b-trusted-ca\") pod \"console-operator-85b454874b-nnwxp\" (UID: \"1cee7771-6381-4fad-8c1e-ad1098fe409b\") " pod="openshift-console-operator/console-operator-85b454874b-nnwxp"
Sep 10 00:50:51.941307 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.941151    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/501b4642-4933-4505-afe4-4446b75ce2ee-serving-cert\") pod \"insights-operator-76bcd6ccf5-kdm6r\" (UID: \"501b4642-4933-4505-afe4-4446b75ce2ee\") " pod="openshift-insights/insights-operator-76bcd6ccf5-kdm6r"
Sep 10 00:50:51.941307 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.941173    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx5vm\" (UniqueName: \"kubernetes.io/projected/501b4642-4933-4505-afe4-4446b75ce2ee-kube-api-access-tx5vm\") pod \"insights-operator-76bcd6ccf5-kdm6r\" (UID: \"501b4642-4933-4505-afe4-4446b75ce2ee\") " pod="openshift-insights/insights-operator-76bcd6ccf5-kdm6r"
Sep 10 00:50:51.941307 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.941208    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/501b4642-4933-4505-afe4-4446b75ce2ee-tmp\") pod \"insights-operator-76bcd6ccf5-kdm6r\" (UID: \"501b4642-4933-4505-afe4-4446b75ce2ee\") " pod="openshift-insights/insights-operator-76bcd6ccf5-kdm6r"
Sep 10 00:50:51.941307 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.941231    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6ebec561-8cb9-40af-92f8-dae61b52833b-serving-cert\") pod \"service-ca-operator-859f94c4b-rpj5t\" (UID: \"6ebec561-8cb9-40af-92f8-dae61b52833b\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rpj5t"
Sep 10 00:50:51.941307 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:51.941241    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 00:50:51.941307 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.941255    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/784a9928-9ed1-4f75-afdb-c42ee2030777-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-4qhvr\" (UID: \"784a9928-9ed1-4f75-afdb-c42ee2030777\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-4qhvr"
Sep 10 00:50:51.941307 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.941291    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b71eb054-e5f2-4f16-b9a3-1253c28778ca-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-gzrtx\" (UID: \"b71eb054-e5f2-4f16-b9a3-1253c28778ca\") " pod="openshift-network-console/networking-console-plugin-79977f9844-gzrtx"
Sep 10 00:50:51.941836 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.941322    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/501b4642-4933-4505-afe4-4446b75ce2ee-snapshots\") pod \"insights-operator-76bcd6ccf5-kdm6r\" (UID: \"501b4642-4933-4505-afe4-4446b75ce2ee\") " pod="openshift-insights/insights-operator-76bcd6ccf5-kdm6r"
Sep 10 00:50:51.941836 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.941346    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/501b4642-4933-4505-afe4-4446b75ce2ee-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-kdm6r\" (UID: \"501b4642-4933-4505-afe4-4446b75ce2ee\") " pod="openshift-insights/insights-operator-76bcd6ccf5-kdm6r"
Sep 10 00:50:51.941836 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:51.941425    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/784a9928-9ed1-4f75-afdb-c42ee2030777-cluster-monitoring-operator-tls podName:784a9928-9ed1-4f75-afdb-c42ee2030777 nodeName:}" failed. No retries permitted until 2026-09-10 00:50:52.441399348 +0000 UTC m=+68.938699115 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/784a9928-9ed1-4f75-afdb-c42ee2030777-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-4qhvr" (UID: "784a9928-9ed1-4f75-afdb-c42ee2030777") : secret "cluster-monitoring-operator-tls" not found
Sep 10 00:50:51.941836 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.941605    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b71eb054-e5f2-4f16-b9a3-1253c28778ca-nginx-conf\") pod \"networking-console-plugin-79977f9844-gzrtx\" (UID: \"b71eb054-e5f2-4f16-b9a3-1253c28778ca\") " pod="openshift-network-console/networking-console-plugin-79977f9844-gzrtx"
Sep 10 00:50:51.941836 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:51.941685    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 00:50:51.941836 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.941702    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/501b4642-4933-4505-afe4-4446b75ce2ee-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-kdm6r\" (UID: \"501b4642-4933-4505-afe4-4446b75ce2ee\") " pod="openshift-insights/insights-operator-76bcd6ccf5-kdm6r"
Sep 10 00:50:51.941836 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:51.941730    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b71eb054-e5f2-4f16-b9a3-1253c28778ca-networking-console-plugin-cert podName:b71eb054-e5f2-4f16-b9a3-1253c28778ca nodeName:}" failed. No retries permitted until 2026-09-10 00:50:52.441715731 +0000 UTC m=+68.939015288 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b71eb054-e5f2-4f16-b9a3-1253c28778ca-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-gzrtx" (UID: "b71eb054-e5f2-4f16-b9a3-1253c28778ca") : secret "networking-console-plugin-cert" not found
Sep 10 00:50:51.942244 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.942149    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/784a9928-9ed1-4f75-afdb-c42ee2030777-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-4qhvr\" (UID: \"784a9928-9ed1-4f75-afdb-c42ee2030777\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-4qhvr"
Sep 10 00:50:51.942244 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.942160    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1cee7771-6381-4fad-8c1e-ad1098fe409b-trusted-ca\") pod \"console-operator-85b454874b-nnwxp\" (UID: \"1cee7771-6381-4fad-8c1e-ad1098fe409b\") " pod="openshift-console-operator/console-operator-85b454874b-nnwxp"
Sep 10 00:50:51.942244 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.942196    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/501b4642-4933-4505-afe4-4446b75ce2ee-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-kdm6r\" (UID: \"501b4642-4933-4505-afe4-4446b75ce2ee\") " pod="openshift-insights/insights-operator-76bcd6ccf5-kdm6r"
Sep 10 00:50:51.942390 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.942350    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6ebec561-8cb9-40af-92f8-dae61b52833b-config\") pod \"service-ca-operator-859f94c4b-rpj5t\" (UID: \"6ebec561-8cb9-40af-92f8-dae61b52833b\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rpj5t"
Sep 10 00:50:51.942529 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.942504    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/501b4642-4933-4505-afe4-4446b75ce2ee-snapshots\") pod \"insights-operator-76bcd6ccf5-kdm6r\" (UID: \"501b4642-4933-4505-afe4-4446b75ce2ee\") " pod="openshift-insights/insights-operator-76bcd6ccf5-kdm6r"
Sep 10 00:50:51.942913 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.942875    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1cee7771-6381-4fad-8c1e-ad1098fe409b-config\") pod \"console-operator-85b454874b-nnwxp\" (UID: \"1cee7771-6381-4fad-8c1e-ad1098fe409b\") " pod="openshift-console-operator/console-operator-85b454874b-nnwxp"
Sep 10 00:50:51.944594 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.944533    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6ebec561-8cb9-40af-92f8-dae61b52833b-serving-cert\") pod \"service-ca-operator-859f94c4b-rpj5t\" (UID: \"6ebec561-8cb9-40af-92f8-dae61b52833b\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rpj5t"
Sep 10 00:50:51.944910 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.944887    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/501b4642-4933-4505-afe4-4446b75ce2ee-serving-cert\") pod \"insights-operator-76bcd6ccf5-kdm6r\" (UID: \"501b4642-4933-4505-afe4-4446b75ce2ee\") " pod="openshift-insights/insights-operator-76bcd6ccf5-kdm6r"
Sep 10 00:50:51.945455 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.945439    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1cee7771-6381-4fad-8c1e-ad1098fe409b-serving-cert\") pod \"console-operator-85b454874b-nnwxp\" (UID: \"1cee7771-6381-4fad-8c1e-ad1098fe409b\") " pod="openshift-console-operator/console-operator-85b454874b-nnwxp"
Sep 10 00:50:51.957690 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.957664    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg485\" (UniqueName: \"kubernetes.io/projected/1cee7771-6381-4fad-8c1e-ad1098fe409b-kube-api-access-tg485\") pod \"console-operator-85b454874b-nnwxp\" (UID: \"1cee7771-6381-4fad-8c1e-ad1098fe409b\") " pod="openshift-console-operator/console-operator-85b454874b-nnwxp"
Sep 10 00:50:51.958143 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.958116    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hmsh\" (UniqueName: \"kubernetes.io/projected/6ebec561-8cb9-40af-92f8-dae61b52833b-kube-api-access-5hmsh\") pod \"service-ca-operator-859f94c4b-rpj5t\" (UID: \"6ebec561-8cb9-40af-92f8-dae61b52833b\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rpj5t"
Sep 10 00:50:51.961484 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.961460    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx5vm\" (UniqueName: \"kubernetes.io/projected/501b4642-4933-4505-afe4-4446b75ce2ee-kube-api-access-tx5vm\") pod \"insights-operator-76bcd6ccf5-kdm6r\" (UID: \"501b4642-4933-4505-afe4-4446b75ce2ee\") " pod="openshift-insights/insights-operator-76bcd6ccf5-kdm6r"
Sep 10 00:50:51.961759 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:51.961736    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8wrd\" (UniqueName: \"kubernetes.io/projected/784a9928-9ed1-4f75-afdb-c42ee2030777-kube-api-access-p8wrd\") pod \"cluster-monitoring-operator-6cdcc96965-4qhvr\" (UID: \"784a9928-9ed1-4f75-afdb-c42ee2030777\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-4qhvr"
Sep 10 00:50:52.147224 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:52.147188    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rpj5t"
Sep 10 00:50:52.155269 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:52.155240    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-nnwxp"
Sep 10 00:50:52.163273 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:52.163243    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-kdm6r"
Sep 10 00:50:52.446463 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:52.446214    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b71eb054-e5f2-4f16-b9a3-1253c28778ca-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-gzrtx\" (UID: \"b71eb054-e5f2-4f16-b9a3-1253c28778ca\") " pod="openshift-network-console/networking-console-plugin-79977f9844-gzrtx"
Sep 10 00:50:52.446463 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:52.446328    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/784a9928-9ed1-4f75-afdb-c42ee2030777-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-4qhvr\" (UID: \"784a9928-9ed1-4f75-afdb-c42ee2030777\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-4qhvr"
Sep 10 00:50:52.446463 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:52.446461    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 00:50:52.447011 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:52.446516    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 00:50:52.447011 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:52.446594    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b71eb054-e5f2-4f16-b9a3-1253c28778ca-networking-console-plugin-cert podName:b71eb054-e5f2-4f16-b9a3-1253c28778ca nodeName:}" failed. No retries permitted until 2026-09-10 00:50:53.44657322 +0000 UTC m=+69.943872777 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b71eb054-e5f2-4f16-b9a3-1253c28778ca-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-gzrtx" (UID: "b71eb054-e5f2-4f16-b9a3-1253c28778ca") : secret "networking-console-plugin-cert" not found
Sep 10 00:50:52.447011 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:52.446619    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/784a9928-9ed1-4f75-afdb-c42ee2030777-cluster-monitoring-operator-tls podName:784a9928-9ed1-4f75-afdb-c42ee2030777 nodeName:}" failed. No retries permitted until 2026-09-10 00:50:53.446607635 +0000 UTC m=+69.943907208 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/784a9928-9ed1-4f75-afdb-c42ee2030777-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-4qhvr" (UID: "784a9928-9ed1-4f75-afdb-c42ee2030777") : secret "cluster-monitoring-operator-tls" not found
Sep 10 00:50:52.448158 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:52.448078    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-rpj5t"]
Sep 10 00:50:52.449067 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:52.448618    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-nnwxp"]
Sep 10 00:50:52.452585 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:50:52.452558    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cee7771_6381_4fad_8c1e_ad1098fe409b.slice/crio-20ea415287f098df00a7538bec0b45c1c816a5b032e859745e5ea4eb01d5c6d9 WatchSource:0}: Error finding container 20ea415287f098df00a7538bec0b45c1c816a5b032e859745e5ea4eb01d5c6d9: Status 404 returned error can't find the container with id 20ea415287f098df00a7538bec0b45c1c816a5b032e859745e5ea4eb01d5c6d9
Sep 10 00:50:52.453768 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:50:52.453746    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ebec561_8cb9_40af_92f8_dae61b52833b.slice/crio-7d697e44ec57ab947faf4d85cee312e804ec32a9fe71caf0767a72c7ffd95f10 WatchSource:0}: Error finding container 7d697e44ec57ab947faf4d85cee312e804ec32a9fe71caf0767a72c7ffd95f10: Status 404 returned error can't find the container with id 7d697e44ec57ab947faf4d85cee312e804ec32a9fe71caf0767a72c7ffd95f10
Sep 10 00:50:52.478673 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:52.478648    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-kdm6r"]
Sep 10 00:50:52.482141 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:50:52.482120    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod501b4642_4933_4505_afe4_4446b75ce2ee.slice/crio-017fc8541122b7c55b8660c4f49ea5425ca47f3811bd2fc54ae5167ee89da403 WatchSource:0}: Error finding container 017fc8541122b7c55b8660c4f49ea5425ca47f3811bd2fc54ae5167ee89da403: Status 404 returned error can't find the container with id 017fc8541122b7c55b8660c4f49ea5425ca47f3811bd2fc54ae5167ee89da403
Sep 10 00:50:53.373700 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:53.373647    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rpj5t" event={"ID":"6ebec561-8cb9-40af-92f8-dae61b52833b","Type":"ContainerStarted","Data":"7d697e44ec57ab947faf4d85cee312e804ec32a9fe71caf0767a72c7ffd95f10"}
Sep 10 00:50:53.375567 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:53.375528    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-nnwxp" event={"ID":"1cee7771-6381-4fad-8c1e-ad1098fe409b","Type":"ContainerStarted","Data":"20ea415287f098df00a7538bec0b45c1c816a5b032e859745e5ea4eb01d5c6d9"}
Sep 10 00:50:53.377969 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:53.377941    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-kdm6r" event={"ID":"501b4642-4933-4505-afe4-4446b75ce2ee","Type":"ContainerStarted","Data":"017fc8541122b7c55b8660c4f49ea5425ca47f3811bd2fc54ae5167ee89da403"}
Sep 10 00:50:53.454833 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:53.454796    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/784a9928-9ed1-4f75-afdb-c42ee2030777-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-4qhvr\" (UID: \"784a9928-9ed1-4f75-afdb-c42ee2030777\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-4qhvr"
Sep 10 00:50:53.455272 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:53.454876    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b71eb054-e5f2-4f16-b9a3-1253c28778ca-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-gzrtx\" (UID: \"b71eb054-e5f2-4f16-b9a3-1253c28778ca\") " pod="openshift-network-console/networking-console-plugin-79977f9844-gzrtx"
Sep 10 00:50:53.455272 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:53.454976    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 00:50:53.455272 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:53.455053    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 00:50:53.455272 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:53.455072    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/784a9928-9ed1-4f75-afdb-c42ee2030777-cluster-monitoring-operator-tls podName:784a9928-9ed1-4f75-afdb-c42ee2030777 nodeName:}" failed. No retries permitted until 2026-09-10 00:50:55.455050616 +0000 UTC m=+71.952350191 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/784a9928-9ed1-4f75-afdb-c42ee2030777-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-4qhvr" (UID: "784a9928-9ed1-4f75-afdb-c42ee2030777") : secret "cluster-monitoring-operator-tls" not found
Sep 10 00:50:53.455272 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:53.455109    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b71eb054-e5f2-4f16-b9a3-1253c28778ca-networking-console-plugin-cert podName:b71eb054-e5f2-4f16-b9a3-1253c28778ca nodeName:}" failed. No retries permitted until 2026-09-10 00:50:55.45509263 +0000 UTC m=+71.952392199 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b71eb054-e5f2-4f16-b9a3-1253c28778ca-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-gzrtx" (UID: "b71eb054-e5f2-4f16-b9a3-1253c28778ca") : secret "networking-console-plugin-cert" not found
Sep 10 00:50:55.472266 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:55.472221    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/784a9928-9ed1-4f75-afdb-c42ee2030777-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-4qhvr\" (UID: \"784a9928-9ed1-4f75-afdb-c42ee2030777\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-4qhvr"
Sep 10 00:50:55.472626 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:55.472317    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b71eb054-e5f2-4f16-b9a3-1253c28778ca-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-gzrtx\" (UID: \"b71eb054-e5f2-4f16-b9a3-1253c28778ca\") " pod="openshift-network-console/networking-console-plugin-79977f9844-gzrtx"
Sep 10 00:50:55.472626 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:55.472387    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 00:50:55.472626 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:55.472470    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/784a9928-9ed1-4f75-afdb-c42ee2030777-cluster-monitoring-operator-tls podName:784a9928-9ed1-4f75-afdb-c42ee2030777 nodeName:}" failed. No retries permitted until 2026-09-10 00:50:59.472448745 +0000 UTC m=+75.969748326 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/784a9928-9ed1-4f75-afdb-c42ee2030777-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-4qhvr" (UID: "784a9928-9ed1-4f75-afdb-c42ee2030777") : secret "cluster-monitoring-operator-tls" not found
Sep 10 00:50:55.472626 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:55.472470    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 00:50:55.472626 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:55.472516    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b71eb054-e5f2-4f16-b9a3-1253c28778ca-networking-console-plugin-cert podName:b71eb054-e5f2-4f16-b9a3-1253c28778ca nodeName:}" failed. No retries permitted until 2026-09-10 00:50:59.472503065 +0000 UTC m=+75.969802640 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b71eb054-e5f2-4f16-b9a3-1253c28778ca-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-gzrtx" (UID: "b71eb054-e5f2-4f16-b9a3-1253c28778ca") : secret "networking-console-plugin-cert" not found
Sep 10 00:50:58.395828 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:58.395789    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-kdm6r" event={"ID":"501b4642-4933-4505-afe4-4446b75ce2ee","Type":"ContainerStarted","Data":"f53d06aebaf3888dfad70c920fe80324f43d0fa615f69bd72ea1bee5afefc416"}
Sep 10 00:50:58.397388 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:58.397354    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cvjrx" event={"ID":"50d35dde-26ab-4a1a-8f6a-2d663df04bc5","Type":"ContainerStarted","Data":"a5a0955010a3745844592cbd5e7bd38b44dc7dc239d4dcaae01f924291205fe2"}
Sep 10 00:50:58.397514 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:58.397406    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-cvjrx"
Sep 10 00:50:58.398819 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:58.398796    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rpj5t" event={"ID":"6ebec561-8cb9-40af-92f8-dae61b52833b","Type":"ContainerStarted","Data":"0dd2bbf678e5b232cf5c9db69a7c8b48611750e139533938c785698f4fb29015"}
Sep 10 00:50:58.400472 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:58.400455    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nnwxp_1cee7771-6381-4fad-8c1e-ad1098fe409b/console-operator/0.log"
Sep 10 00:50:58.400558 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:58.400486    2563 generic.go:356] "Generic (PLEG): container finished" podID="1cee7771-6381-4fad-8c1e-ad1098fe409b" containerID="9478d724c6c68dae822be3bac0eebbbcae9335936336bfb699f5c246de1ee058" exitCode=255
Sep 10 00:50:58.400558 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:58.400548    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-nnwxp" event={"ID":"1cee7771-6381-4fad-8c1e-ad1098fe409b","Type":"ContainerDied","Data":"9478d724c6c68dae822be3bac0eebbbcae9335936336bfb699f5c246de1ee058"}
Sep 10 00:50:58.400749 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:58.400733    2563 scope.go:117] "RemoveContainer" containerID="9478d724c6c68dae822be3bac0eebbbcae9335936336bfb699f5c246de1ee058"
Sep 10 00:50:58.402100 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:58.402081    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-l4w6p" event={"ID":"00cec203-bdcf-4e75-acfa-29e1b784840e","Type":"ContainerStarted","Data":"3d0b7b5913978a2a12ea5fcec580e8a1f1ad0b7ca7a18a33066e10aedf5f779e"}
Sep 10 00:50:58.414575 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:58.414515    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-kdm6r" podStartSLOduration=2.325545721 podStartE2EDuration="7.414495419s" totalImagesPullingTime="5.088949698s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:50:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:50:58.413380003 +0000 UTC m=+74.910679582" watchObservedRunningTime="2026-09-10 00:50:58.414495419 +0000 UTC m=+74.911794998"
Sep 10 00:50:58.449004 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:58.448936    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-l4w6p" podStartSLOduration=65.941110895 podStartE2EDuration="1m13.448920311s" totalImagesPullingTime="7.507809416s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 00:49:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:50:58.434541462 +0000 UTC m=+74.931841042" watchObservedRunningTime="2026-09-10 00:50:58.448920311 +0000 UTC m=+74.946219889"
Sep 10 00:50:58.449301 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:58.449277    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rpj5t" podStartSLOduration=2.2991884750000002 podStartE2EDuration="7.449271s" totalImagesPullingTime="5.150082525s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:50:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:50:58.448491176 +0000 UTC m=+74.945790751" watchObservedRunningTime="2026-09-10 00:50:58.449271 +0000 UTC m=+74.946570580"
Sep 10 00:50:58.481144 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:58.481085    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-cvjrx" podStartSLOduration=66.290724215 podStartE2EDuration="1m13.481065234s" totalImagesPullingTime="7.190341019s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:49:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:50:58.480349753 +0000 UTC m=+74.977649333" watchObservedRunningTime="2026-09-10 00:50:58.481065234 +0000 UTC m=+74.978364857"
Sep 10 00:50:58.982613 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:58.982579    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-k9rtv"]
Sep 10 00:50:58.985351 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:58.985328    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-k9rtv"
Sep 10 00:50:58.988154 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:58.988129    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 10 00:50:58.988281 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:58.988129    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 10 00:50:58.988281 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:58.988129    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rgq8h\""
Sep 10 00:50:58.999432 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:58.999403    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-k9rtv"]
Sep 10 00:50:59.103640 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:59.103597    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d794c501-ec2b-419a-aefa-b61f7dedad8e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k9rtv\" (UID: \"d794c501-ec2b-419a-aefa-b61f7dedad8e\") " pod="openshift-insights/insights-runtime-extractor-k9rtv"
Sep 10 00:50:59.103640 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:59.103641    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d794c501-ec2b-419a-aefa-b61f7dedad8e-data-volume\") pod \"insights-runtime-extractor-k9rtv\" (UID: \"d794c501-ec2b-419a-aefa-b61f7dedad8e\") " pod="openshift-insights/insights-runtime-extractor-k9rtv"
Sep 10 00:50:59.103860 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:59.103667    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d794c501-ec2b-419a-aefa-b61f7dedad8e-crio-socket\") pod \"insights-runtime-extractor-k9rtv\" (UID: \"d794c501-ec2b-419a-aefa-b61f7dedad8e\") " pod="openshift-insights/insights-runtime-extractor-k9rtv"
Sep 10 00:50:59.103860 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:59.103709    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgjsz\" (UniqueName: \"kubernetes.io/projected/d794c501-ec2b-419a-aefa-b61f7dedad8e-kube-api-access-bgjsz\") pod \"insights-runtime-extractor-k9rtv\" (UID: \"d794c501-ec2b-419a-aefa-b61f7dedad8e\") " pod="openshift-insights/insights-runtime-extractor-k9rtv"
Sep 10 00:50:59.103860 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:59.103742    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d794c501-ec2b-419a-aefa-b61f7dedad8e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k9rtv\" (UID: \"d794c501-ec2b-419a-aefa-b61f7dedad8e\") " pod="openshift-insights/insights-runtime-extractor-k9rtv"
Sep 10 00:50:59.204579 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:59.204536    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d794c501-ec2b-419a-aefa-b61f7dedad8e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k9rtv\" (UID: \"d794c501-ec2b-419a-aefa-b61f7dedad8e\") " pod="openshift-insights/insights-runtime-extractor-k9rtv"
Sep 10 00:50:59.204579 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:59.204583    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d794c501-ec2b-419a-aefa-b61f7dedad8e-data-volume\") pod \"insights-runtime-extractor-k9rtv\" (UID: \"d794c501-ec2b-419a-aefa-b61f7dedad8e\") " pod="openshift-insights/insights-runtime-extractor-k9rtv"
Sep 10 00:50:59.204818 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:59.204618    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d794c501-ec2b-419a-aefa-b61f7dedad8e-crio-socket\") pod \"insights-runtime-extractor-k9rtv\" (UID: \"d794c501-ec2b-419a-aefa-b61f7dedad8e\") " pod="openshift-insights/insights-runtime-extractor-k9rtv"
Sep 10 00:50:59.204818 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:59.204649    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgjsz\" (UniqueName: \"kubernetes.io/projected/d794c501-ec2b-419a-aefa-b61f7dedad8e-kube-api-access-bgjsz\") pod \"insights-runtime-extractor-k9rtv\" (UID: \"d794c501-ec2b-419a-aefa-b61f7dedad8e\") " pod="openshift-insights/insights-runtime-extractor-k9rtv"
Sep 10 00:50:59.204818 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:59.204701    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d794c501-ec2b-419a-aefa-b61f7dedad8e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k9rtv\" (UID: \"d794c501-ec2b-419a-aefa-b61f7dedad8e\") " pod="openshift-insights/insights-runtime-extractor-k9rtv"
Sep 10 00:50:59.204818 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:59.204707    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 00:50:59.204818 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:59.204763    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 00:50:59.204818 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:59.204770    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d794c501-ec2b-419a-aefa-b61f7dedad8e-insights-runtime-extractor-tls podName:d794c501-ec2b-419a-aefa-b61f7dedad8e nodeName:}" failed. No retries permitted until 2026-09-10 00:50:59.704752701 +0000 UTC m=+76.202052277 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d794c501-ec2b-419a-aefa-b61f7dedad8e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k9rtv" (UID: "d794c501-ec2b-419a-aefa-b61f7dedad8e") : secret "insights-runtime-extractor-tls" not found
Sep 10 00:50:59.204818 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:59.204775    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d794c501-ec2b-419a-aefa-b61f7dedad8e-crio-socket\") pod \"insights-runtime-extractor-k9rtv\" (UID: \"d794c501-ec2b-419a-aefa-b61f7dedad8e\") " pod="openshift-insights/insights-runtime-extractor-k9rtv"
Sep 10 00:50:59.204818 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:59.204806    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d794c501-ec2b-419a-aefa-b61f7dedad8e-kube-rbac-proxy-cm podName:d794c501-ec2b-419a-aefa-b61f7dedad8e nodeName:}" failed. No retries permitted until 2026-09-10 00:50:59.704794298 +0000 UTC m=+76.202093869 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d794c501-ec2b-419a-aefa-b61f7dedad8e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k9rtv" (UID: "d794c501-ec2b-419a-aefa-b61f7dedad8e") : configmap "kube-rbac-proxy" not found
Sep 10 00:50:59.205127 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:59.204901    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d794c501-ec2b-419a-aefa-b61f7dedad8e-data-volume\") pod \"insights-runtime-extractor-k9rtv\" (UID: \"d794c501-ec2b-419a-aefa-b61f7dedad8e\") " pod="openshift-insights/insights-runtime-extractor-k9rtv"
Sep 10 00:50:59.215302 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:59.215271    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgjsz\" (UniqueName: \"kubernetes.io/projected/d794c501-ec2b-419a-aefa-b61f7dedad8e-kube-api-access-bgjsz\") pod \"insights-runtime-extractor-k9rtv\" (UID: \"d794c501-ec2b-419a-aefa-b61f7dedad8e\") " pod="openshift-insights/insights-runtime-extractor-k9rtv"
Sep 10 00:50:59.407577 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:59.407546    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nnwxp_1cee7771-6381-4fad-8c1e-ad1098fe409b/console-operator/1.log"
Sep 10 00:50:59.408039 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:59.408021    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nnwxp_1cee7771-6381-4fad-8c1e-ad1098fe409b/console-operator/0.log"
Sep 10 00:50:59.408111 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:59.408066    2563 generic.go:356] "Generic (PLEG): container finished" podID="1cee7771-6381-4fad-8c1e-ad1098fe409b" containerID="b2388a05099941d19f12cc064d3b19284b30e3f064bf1ae701e297d5869d5ccd" exitCode=255
Sep 10 00:50:59.408187 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:59.408164    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-nnwxp" event={"ID":"1cee7771-6381-4fad-8c1e-ad1098fe409b","Type":"ContainerDied","Data":"b2388a05099941d19f12cc064d3b19284b30e3f064bf1ae701e297d5869d5ccd"}
Sep 10 00:50:59.408246 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:59.408229    2563 scope.go:117] "RemoveContainer" containerID="9478d724c6c68dae822be3bac0eebbbcae9335936336bfb699f5c246de1ee058"
Sep 10 00:50:59.408470 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:59.408441    2563 scope.go:117] "RemoveContainer" containerID="b2388a05099941d19f12cc064d3b19284b30e3f064bf1ae701e297d5869d5ccd"
Sep 10 00:50:59.408658 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:59.408633    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-nnwxp_openshift-console-operator(1cee7771-6381-4fad-8c1e-ad1098fe409b)\"" pod="openshift-console-operator/console-operator-85b454874b-nnwxp" podUID="1cee7771-6381-4fad-8c1e-ad1098fe409b"
Sep 10 00:50:59.507179 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:59.507138    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/784a9928-9ed1-4f75-afdb-c42ee2030777-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-4qhvr\" (UID: \"784a9928-9ed1-4f75-afdb-c42ee2030777\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-4qhvr"
Sep 10 00:50:59.507379 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:59.507270    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 00:50:59.507379 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:59.507344    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/784a9928-9ed1-4f75-afdb-c42ee2030777-cluster-monitoring-operator-tls podName:784a9928-9ed1-4f75-afdb-c42ee2030777 nodeName:}" failed. No retries permitted until 2026-09-10 00:51:07.507324481 +0000 UTC m=+84.004624055 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/784a9928-9ed1-4f75-afdb-c42ee2030777-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-4qhvr" (UID: "784a9928-9ed1-4f75-afdb-c42ee2030777") : secret "cluster-monitoring-operator-tls" not found
Sep 10 00:50:59.507752 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:59.507717    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b71eb054-e5f2-4f16-b9a3-1253c28778ca-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-gzrtx\" (UID: \"b71eb054-e5f2-4f16-b9a3-1253c28778ca\") " pod="openshift-network-console/networking-console-plugin-79977f9844-gzrtx"
Sep 10 00:50:59.507859 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:59.507840    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 00:50:59.507936 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:59.507923    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b71eb054-e5f2-4f16-b9a3-1253c28778ca-networking-console-plugin-cert podName:b71eb054-e5f2-4f16-b9a3-1253c28778ca nodeName:}" failed. No retries permitted until 2026-09-10 00:51:07.507903649 +0000 UTC m=+84.005203220 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b71eb054-e5f2-4f16-b9a3-1253c28778ca-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-gzrtx" (UID: "b71eb054-e5f2-4f16-b9a3-1253c28778ca") : secret "networking-console-plugin-cert" not found
Sep 10 00:50:59.710223 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:59.710133    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d794c501-ec2b-419a-aefa-b61f7dedad8e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k9rtv\" (UID: \"d794c501-ec2b-419a-aefa-b61f7dedad8e\") " pod="openshift-insights/insights-runtime-extractor-k9rtv"
Sep 10 00:50:59.710223 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:50:59.710213    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d794c501-ec2b-419a-aefa-b61f7dedad8e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k9rtv\" (UID: \"d794c501-ec2b-419a-aefa-b61f7dedad8e\") " pod="openshift-insights/insights-runtime-extractor-k9rtv"
Sep 10 00:50:59.710409 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:59.710270    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 00:50:59.710409 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:59.710336    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 00:50:59.710409 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:59.710341    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d794c501-ec2b-419a-aefa-b61f7dedad8e-kube-rbac-proxy-cm podName:d794c501-ec2b-419a-aefa-b61f7dedad8e nodeName:}" failed. No retries permitted until 2026-09-10 00:51:00.710324985 +0000 UTC m=+77.207624542 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d794c501-ec2b-419a-aefa-b61f7dedad8e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k9rtv" (UID: "d794c501-ec2b-419a-aefa-b61f7dedad8e") : configmap "kube-rbac-proxy" not found
Sep 10 00:50:59.710409 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:50:59.710383    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d794c501-ec2b-419a-aefa-b61f7dedad8e-insights-runtime-extractor-tls podName:d794c501-ec2b-419a-aefa-b61f7dedad8e nodeName:}" failed. No retries permitted until 2026-09-10 00:51:00.710372535 +0000 UTC m=+77.207672093 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d794c501-ec2b-419a-aefa-b61f7dedad8e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k9rtv" (UID: "d794c501-ec2b-419a-aefa-b61f7dedad8e") : secret "insights-runtime-extractor-tls" not found
Sep 10 00:51:00.413734 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:00.413705    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nnwxp_1cee7771-6381-4fad-8c1e-ad1098fe409b/console-operator/1.log"
Sep 10 00:51:00.414274 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:00.414068    2563 scope.go:117] "RemoveContainer" containerID="b2388a05099941d19f12cc064d3b19284b30e3f064bf1ae701e297d5869d5ccd"
Sep 10 00:51:00.414274 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:51:00.414224    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-nnwxp_openshift-console-operator(1cee7771-6381-4fad-8c1e-ad1098fe409b)\"" pod="openshift-console-operator/console-operator-85b454874b-nnwxp" podUID="1cee7771-6381-4fad-8c1e-ad1098fe409b"
Sep 10 00:51:00.720903 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:00.720797    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d794c501-ec2b-419a-aefa-b61f7dedad8e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k9rtv\" (UID: \"d794c501-ec2b-419a-aefa-b61f7dedad8e\") " pod="openshift-insights/insights-runtime-extractor-k9rtv"
Sep 10 00:51:00.721099 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:00.720905    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d794c501-ec2b-419a-aefa-b61f7dedad8e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k9rtv\" (UID: \"d794c501-ec2b-419a-aefa-b61f7dedad8e\") " pod="openshift-insights/insights-runtime-extractor-k9rtv"
Sep 10 00:51:00.721099 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:51:00.720949    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 00:51:00.721099 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:51:00.721032    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 00:51:00.721099 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:51:00.721050    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d794c501-ec2b-419a-aefa-b61f7dedad8e-insights-runtime-extractor-tls podName:d794c501-ec2b-419a-aefa-b61f7dedad8e nodeName:}" failed. No retries permitted until 2026-09-10 00:51:02.721028645 +0000 UTC m=+79.218328223 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d794c501-ec2b-419a-aefa-b61f7dedad8e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k9rtv" (UID: "d794c501-ec2b-419a-aefa-b61f7dedad8e") : secret "insights-runtime-extractor-tls" not found
Sep 10 00:51:00.721099 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:51:00.721092    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d794c501-ec2b-419a-aefa-b61f7dedad8e-kube-rbac-proxy-cm podName:d794c501-ec2b-419a-aefa-b61f7dedad8e nodeName:}" failed. No retries permitted until 2026-09-10 00:51:02.721079741 +0000 UTC m=+79.218379304 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d794c501-ec2b-419a-aefa-b61f7dedad8e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k9rtv" (UID: "d794c501-ec2b-419a-aefa-b61f7dedad8e") : configmap "kube-rbac-proxy" not found
Sep 10 00:51:01.147378 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:01.147352    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-p5zpt_27351a7f-151c-4d9a-bddd-4ebaa3a42ade/dns-node-resolver/0.log"
Sep 10 00:51:01.947657 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:01.947630    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tr2kj_8686a32b-9d47-4866-aae0-89f0b6575ff3/node-ca/0.log"
Sep 10 00:51:02.155685 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:02.155643    2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-nnwxp"
Sep 10 00:51:02.155885 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:02.155712    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-nnwxp"
Sep 10 00:51:02.156175 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:02.156155    2563 scope.go:117] "RemoveContainer" containerID="b2388a05099941d19f12cc064d3b19284b30e3f064bf1ae701e297d5869d5ccd"
Sep 10 00:51:02.156362 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:51:02.156341    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-nnwxp_openshift-console-operator(1cee7771-6381-4fad-8c1e-ad1098fe409b)\"" pod="openshift-console-operator/console-operator-85b454874b-nnwxp" podUID="1cee7771-6381-4fad-8c1e-ad1098fe409b"
Sep 10 00:51:02.735641 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:02.735605    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d794c501-ec2b-419a-aefa-b61f7dedad8e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k9rtv\" (UID: \"d794c501-ec2b-419a-aefa-b61f7dedad8e\") " pod="openshift-insights/insights-runtime-extractor-k9rtv"
Sep 10 00:51:02.735832 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:02.735677    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d794c501-ec2b-419a-aefa-b61f7dedad8e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k9rtv\" (UID: \"d794c501-ec2b-419a-aefa-b61f7dedad8e\") " pod="openshift-insights/insights-runtime-extractor-k9rtv"
Sep 10 00:51:02.735832 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:51:02.735730    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 00:51:02.735832 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:51:02.735767    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 00:51:02.735832 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:51:02.735806    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d794c501-ec2b-419a-aefa-b61f7dedad8e-kube-rbac-proxy-cm podName:d794c501-ec2b-419a-aefa-b61f7dedad8e nodeName:}" failed. No retries permitted until 2026-09-10 00:51:06.735790056 +0000 UTC m=+83.233089618 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d794c501-ec2b-419a-aefa-b61f7dedad8e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k9rtv" (UID: "d794c501-ec2b-419a-aefa-b61f7dedad8e") : configmap "kube-rbac-proxy" not found
Sep 10 00:51:02.735832 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:51:02.735828    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d794c501-ec2b-419a-aefa-b61f7dedad8e-insights-runtime-extractor-tls podName:d794c501-ec2b-419a-aefa-b61f7dedad8e nodeName:}" failed. No retries permitted until 2026-09-10 00:51:06.735816673 +0000 UTC m=+83.233116229 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d794c501-ec2b-419a-aefa-b61f7dedad8e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k9rtv" (UID: "d794c501-ec2b-419a-aefa-b61f7dedad8e") : secret "insights-runtime-extractor-tls" not found
Sep 10 00:51:02.947624 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:02.947597    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sgpwp_96504ff2-84b9-442b-a77e-254cbe9d8e5a/serve-healthcheck-canary/0.log"
Sep 10 00:51:06.769349 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:06.769195    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d794c501-ec2b-419a-aefa-b61f7dedad8e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k9rtv\" (UID: \"d794c501-ec2b-419a-aefa-b61f7dedad8e\") " pod="openshift-insights/insights-runtime-extractor-k9rtv"
Sep 10 00:51:06.769349 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:06.769300    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d794c501-ec2b-419a-aefa-b61f7dedad8e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k9rtv\" (UID: \"d794c501-ec2b-419a-aefa-b61f7dedad8e\") " pod="openshift-insights/insights-runtime-extractor-k9rtv"
Sep 10 00:51:06.769770 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:51:06.769357    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 00:51:06.769770 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:51:06.769413    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 00:51:06.769770 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:51:06.769434    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d794c501-ec2b-419a-aefa-b61f7dedad8e-insights-runtime-extractor-tls podName:d794c501-ec2b-419a-aefa-b61f7dedad8e nodeName:}" failed. No retries permitted until 2026-09-10 00:51:14.76941713 +0000 UTC m=+91.266716687 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d794c501-ec2b-419a-aefa-b61f7dedad8e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k9rtv" (UID: "d794c501-ec2b-419a-aefa-b61f7dedad8e") : secret "insights-runtime-extractor-tls" not found
Sep 10 00:51:06.769770 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:51:06.769473    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d794c501-ec2b-419a-aefa-b61f7dedad8e-kube-rbac-proxy-cm podName:d794c501-ec2b-419a-aefa-b61f7dedad8e nodeName:}" failed. No retries permitted until 2026-09-10 00:51:14.769460565 +0000 UTC m=+91.266760128 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d794c501-ec2b-419a-aefa-b61f7dedad8e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k9rtv" (UID: "d794c501-ec2b-419a-aefa-b61f7dedad8e") : configmap "kube-rbac-proxy" not found
Sep 10 00:51:07.574906 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:07.574866    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/784a9928-9ed1-4f75-afdb-c42ee2030777-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-4qhvr\" (UID: \"784a9928-9ed1-4f75-afdb-c42ee2030777\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-4qhvr"
Sep 10 00:51:07.575152 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:07.574959    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b71eb054-e5f2-4f16-b9a3-1253c28778ca-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-gzrtx\" (UID: \"b71eb054-e5f2-4f16-b9a3-1253c28778ca\") " pod="openshift-network-console/networking-console-plugin-79977f9844-gzrtx"
Sep 10 00:51:07.575152 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:51:07.575047    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 00:51:07.575152 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:51:07.575082    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 00:51:07.575152 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:51:07.575126    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/784a9928-9ed1-4f75-afdb-c42ee2030777-cluster-monitoring-operator-tls podName:784a9928-9ed1-4f75-afdb-c42ee2030777 nodeName:}" failed. No retries permitted until 2026-09-10 00:51:23.575111796 +0000 UTC m=+100.072411353 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/784a9928-9ed1-4f75-afdb-c42ee2030777-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-4qhvr" (UID: "784a9928-9ed1-4f75-afdb-c42ee2030777") : secret "cluster-monitoring-operator-tls" not found
Sep 10 00:51:07.575152 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:51:07.575141    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b71eb054-e5f2-4f16-b9a3-1253c28778ca-networking-console-plugin-cert podName:b71eb054-e5f2-4f16-b9a3-1253c28778ca nodeName:}" failed. No retries permitted until 2026-09-10 00:51:23.575135179 +0000 UTC m=+100.072434737 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b71eb054-e5f2-4f16-b9a3-1253c28778ca-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-gzrtx" (UID: "b71eb054-e5f2-4f16-b9a3-1253c28778ca") : secret "networking-console-plugin-cert" not found
Sep 10 00:51:14.833691 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:14.833641    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d794c501-ec2b-419a-aefa-b61f7dedad8e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k9rtv\" (UID: \"d794c501-ec2b-419a-aefa-b61f7dedad8e\") " pod="openshift-insights/insights-runtime-extractor-k9rtv"
Sep 10 00:51:14.834128 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:14.833733    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d794c501-ec2b-419a-aefa-b61f7dedad8e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k9rtv\" (UID: \"d794c501-ec2b-419a-aefa-b61f7dedad8e\") " pod="openshift-insights/insights-runtime-extractor-k9rtv"
Sep 10 00:51:14.834128 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:51:14.833797    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 00:51:14.834128 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:51:14.833867    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d794c501-ec2b-419a-aefa-b61f7dedad8e-insights-runtime-extractor-tls podName:d794c501-ec2b-419a-aefa-b61f7dedad8e nodeName:}" failed. No retries permitted until 2026-09-10 00:51:30.833847362 +0000 UTC m=+107.331146940 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d794c501-ec2b-419a-aefa-b61f7dedad8e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k9rtv" (UID: "d794c501-ec2b-419a-aefa-b61f7dedad8e") : secret "insights-runtime-extractor-tls" not found
Sep 10 00:51:14.834261 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:14.834235    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d794c501-ec2b-419a-aefa-b61f7dedad8e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k9rtv\" (UID: \"d794c501-ec2b-419a-aefa-b61f7dedad8e\") " pod="openshift-insights/insights-runtime-extractor-k9rtv"
Sep 10 00:51:16.076500 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:16.076465    2563 scope.go:117] "RemoveContainer" containerID="b2388a05099941d19f12cc064d3b19284b30e3f064bf1ae701e297d5869d5ccd"
Sep 10 00:51:16.472079 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:16.472047    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nnwxp_1cee7771-6381-4fad-8c1e-ad1098fe409b/console-operator/1.log"
Sep 10 00:51:16.472277 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:16.472125    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-nnwxp" event={"ID":"1cee7771-6381-4fad-8c1e-ad1098fe409b","Type":"ContainerStarted","Data":"7d55a6d72d1e07ce6efe8e623bc1c0b482751b8949a12c76f8e491fc1d7bfd98"}
Sep 10 00:51:16.472504 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:16.472483    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-nnwxp"
Sep 10 00:51:16.487894 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:16.487852    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-nnwxp" podStartSLOduration=20.340760933 podStartE2EDuration="25.487838205s" totalImagesPullingTime="5.147077272s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:50:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:51:16.487289083 +0000 UTC m=+92.984588663" watchObservedRunningTime="2026-09-10 00:51:16.487838205 +0000 UTC m=+92.985137784"
Sep 10 00:51:16.763352 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:16.763269    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-nnwxp"
Sep 10 00:51:22.292849 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:22.292761    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-registry-tls\") pod \"image-registry-b54b88b4b-q42sn\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") " pod="openshift-image-registry/image-registry-b54b88b4b-q42sn"
Sep 10 00:51:22.295252 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:22.295229    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-registry-tls\") pod \"image-registry-b54b88b4b-q42sn\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") " pod="openshift-image-registry/image-registry-b54b88b4b-q42sn"
Sep 10 00:51:22.494534 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:22.494506    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/58138b99-90e8-4b0a-89bf-891604fbaca8-metrics-tls\") pod \"dns-default-kw2lh\" (UID: \"58138b99-90e8-4b0a-89bf-891604fbaca8\") " pod="openshift-dns/dns-default-kw2lh"
Sep 10 00:51:22.496746 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:22.496716    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/58138b99-90e8-4b0a-89bf-891604fbaca8-metrics-tls\") pod \"dns-default-kw2lh\" (UID: \"58138b99-90e8-4b0a-89bf-891604fbaca8\") " pod="openshift-dns/dns-default-kw2lh"
Sep 10 00:51:22.580496 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:22.580416    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tmd2k\""
Sep 10 00:51:22.588588 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:22.588566    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b54b88b4b-q42sn"
Sep 10 00:51:22.753516 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:22.753482    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fhnl9\""
Sep 10 00:51:22.761548 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:22.761526    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kw2lh"
Sep 10 00:51:22.860380 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:22.860348    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b54b88b4b-q42sn"]
Sep 10 00:51:22.863584 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:51:22.863561    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cf5ecf8_4fd5_4c6e_99ea_4015ff995450.slice/crio-fd5a45bc81e6eb5906e7e116a8b631f97b081d3ed832432eca75c006dea96a76 WatchSource:0}: Error finding container fd5a45bc81e6eb5906e7e116a8b631f97b081d3ed832432eca75c006dea96a76: Status 404 returned error can't find the container with id fd5a45bc81e6eb5906e7e116a8b631f97b081d3ed832432eca75c006dea96a76
Sep 10 00:51:23.030576 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:23.030541    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kw2lh"]
Sep 10 00:51:23.034040 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:51:23.034013    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58138b99_90e8_4b0a_89bf_891604fbaca8.slice/crio-4483131c1bc58df8a4f11e68f39ef65aef32e0f3c9880ec2e63d55bdb04bf144 WatchSource:0}: Error finding container 4483131c1bc58df8a4f11e68f39ef65aef32e0f3c9880ec2e63d55bdb04bf144: Status 404 returned error can't find the container with id 4483131c1bc58df8a4f11e68f39ef65aef32e0f3c9880ec2e63d55bdb04bf144
Sep 10 00:51:23.494481 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:23.494444    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b54b88b4b-q42sn" event={"ID":"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450","Type":"ContainerStarted","Data":"bd382025971652e50667c2aee38fe063dcb042c3359bc8940e3d6818659bbbe1"}
Sep 10 00:51:23.494913 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:23.494495    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b54b88b4b-q42sn" event={"ID":"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450","Type":"ContainerStarted","Data":"fd5a45bc81e6eb5906e7e116a8b631f97b081d3ed832432eca75c006dea96a76"}
Sep 10 00:51:23.494913 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:23.494535    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-b54b88b4b-q42sn"
Sep 10 00:51:23.495648 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:23.495627    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kw2lh" event={"ID":"58138b99-90e8-4b0a-89bf-891604fbaca8","Type":"ContainerStarted","Data":"4483131c1bc58df8a4f11e68f39ef65aef32e0f3c9880ec2e63d55bdb04bf144"}
Sep 10 00:51:23.515645 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:23.515598    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-b54b88b4b-q42sn" podStartSLOduration=98.515583592 podStartE2EDuration="1m38.515583592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:49:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:51:23.514541269 +0000 UTC m=+100.011840848" watchObservedRunningTime="2026-09-10 00:51:23.515583592 +0000 UTC m=+100.012883170"
Sep 10 00:51:23.605168 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:23.605136    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/784a9928-9ed1-4f75-afdb-c42ee2030777-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-4qhvr\" (UID: \"784a9928-9ed1-4f75-afdb-c42ee2030777\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-4qhvr"
Sep 10 00:51:23.605344 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:23.605215    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b71eb054-e5f2-4f16-b9a3-1253c28778ca-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-gzrtx\" (UID: \"b71eb054-e5f2-4f16-b9a3-1253c28778ca\") " pod="openshift-network-console/networking-console-plugin-79977f9844-gzrtx"
Sep 10 00:51:23.607704 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:23.607676    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/784a9928-9ed1-4f75-afdb-c42ee2030777-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-4qhvr\" (UID: \"784a9928-9ed1-4f75-afdb-c42ee2030777\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-4qhvr"
Sep 10 00:51:23.607810 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:23.607792    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b71eb054-e5f2-4f16-b9a3-1253c28778ca-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-gzrtx\" (UID: \"b71eb054-e5f2-4f16-b9a3-1253c28778ca\") " pod="openshift-network-console/networking-console-plugin-79977f9844-gzrtx"
Sep 10 00:51:23.629746 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:23.629714    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-gzrtx"
Sep 10 00:51:23.668902 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:23.668863    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-4qhvr"
Sep 10 00:51:23.910418 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:23.910385    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-gzrtx"]
Sep 10 00:51:23.913809 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:51:23.913784    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb71eb054_e5f2_4f16_b9a3_1253c28778ca.slice/crio-0f1564efcb2c5fd6fd65457139e9218904aca9acfb6b6505420eb1c018144588 WatchSource:0}: Error finding container 0f1564efcb2c5fd6fd65457139e9218904aca9acfb6b6505420eb1c018144588: Status 404 returned error can't find the container with id 0f1564efcb2c5fd6fd65457139e9218904aca9acfb6b6505420eb1c018144588
Sep 10 00:51:23.953658 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:23.953619    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-4qhvr"]
Sep 10 00:51:23.956658 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:51:23.956635    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod784a9928_9ed1_4f75_afdb_c42ee2030777.slice/crio-71458dd5075ee1d936c8b371911e6e26031d3486f85f20ce3e6bdc7e619473a7 WatchSource:0}: Error finding container 71458dd5075ee1d936c8b371911e6e26031d3486f85f20ce3e6bdc7e619473a7: Status 404 returned error can't find the container with id 71458dd5075ee1d936c8b371911e6e26031d3486f85f20ce3e6bdc7e619473a7
Sep 10 00:51:24.501522 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:24.501486    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-4qhvr" event={"ID":"784a9928-9ed1-4f75-afdb-c42ee2030777","Type":"ContainerStarted","Data":"71458dd5075ee1d936c8b371911e6e26031d3486f85f20ce3e6bdc7e619473a7"}
Sep 10 00:51:24.502830 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:24.502799    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-gzrtx" event={"ID":"b71eb054-e5f2-4f16-b9a3-1253c28778ca","Type":"ContainerStarted","Data":"0f1564efcb2c5fd6fd65457139e9218904aca9acfb6b6505420eb1c018144588"}
Sep 10 00:51:24.785770 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:24.785686    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c4ddcd77f-hc9bn"]
Sep 10 00:51:24.807102 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:24.807068    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c4ddcd77f-hc9bn"]
Sep 10 00:51:24.807281 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:24.807194    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c4ddcd77f-hc9bn"
Sep 10 00:51:24.813045 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:24.813018    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 10 00:51:24.813241 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:24.813220    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 10 00:51:24.813429 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:24.813412    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 10 00:51:24.813670 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:24.813636    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 10 00:51:24.919431 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:24.919394    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/631da3ce-c11f-492d-9928-ed81d8d5d166-klusterlet-config\") pod \"klusterlet-addon-workmgr-7c4ddcd77f-hc9bn\" (UID: \"631da3ce-c11f-492d-9928-ed81d8d5d166\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c4ddcd77f-hc9bn"
Sep 10 00:51:24.919584 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:24.919516    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/631da3ce-c11f-492d-9928-ed81d8d5d166-tmp\") pod \"klusterlet-addon-workmgr-7c4ddcd77f-hc9bn\" (UID: \"631da3ce-c11f-492d-9928-ed81d8d5d166\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c4ddcd77f-hc9bn"
Sep 10 00:51:24.919584 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:24.919557    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2kzx\" (UniqueName: \"kubernetes.io/projected/631da3ce-c11f-492d-9928-ed81d8d5d166-kube-api-access-z2kzx\") pod \"klusterlet-addon-workmgr-7c4ddcd77f-hc9bn\" (UID: \"631da3ce-c11f-492d-9928-ed81d8d5d166\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c4ddcd77f-hc9bn"
Sep 10 00:51:25.020331 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:25.020300    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/631da3ce-c11f-492d-9928-ed81d8d5d166-tmp\") pod \"klusterlet-addon-workmgr-7c4ddcd77f-hc9bn\" (UID: \"631da3ce-c11f-492d-9928-ed81d8d5d166\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c4ddcd77f-hc9bn"
Sep 10 00:51:25.020331 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:25.020334    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2kzx\" (UniqueName: \"kubernetes.io/projected/631da3ce-c11f-492d-9928-ed81d8d5d166-kube-api-access-z2kzx\") pod \"klusterlet-addon-workmgr-7c4ddcd77f-hc9bn\" (UID: \"631da3ce-c11f-492d-9928-ed81d8d5d166\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c4ddcd77f-hc9bn"
Sep 10 00:51:25.020542 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:25.020414    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/631da3ce-c11f-492d-9928-ed81d8d5d166-klusterlet-config\") pod \"klusterlet-addon-workmgr-7c4ddcd77f-hc9bn\" (UID: \"631da3ce-c11f-492d-9928-ed81d8d5d166\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c4ddcd77f-hc9bn"
Sep 10 00:51:25.020763 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:25.020739    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/631da3ce-c11f-492d-9928-ed81d8d5d166-tmp\") pod \"klusterlet-addon-workmgr-7c4ddcd77f-hc9bn\" (UID: \"631da3ce-c11f-492d-9928-ed81d8d5d166\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c4ddcd77f-hc9bn"
Sep 10 00:51:25.023294 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:25.023269    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/631da3ce-c11f-492d-9928-ed81d8d5d166-klusterlet-config\") pod \"klusterlet-addon-workmgr-7c4ddcd77f-hc9bn\" (UID: \"631da3ce-c11f-492d-9928-ed81d8d5d166\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c4ddcd77f-hc9bn"
Sep 10 00:51:25.029687 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:25.029663    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2kzx\" (UniqueName: \"kubernetes.io/projected/631da3ce-c11f-492d-9928-ed81d8d5d166-kube-api-access-z2kzx\") pod \"klusterlet-addon-workmgr-7c4ddcd77f-hc9bn\" (UID: \"631da3ce-c11f-492d-9928-ed81d8d5d166\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c4ddcd77f-hc9bn"
Sep 10 00:51:25.121618 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:25.121571    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c4ddcd77f-hc9bn"
Sep 10 00:51:26.781959 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:26.781931    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c4ddcd77f-hc9bn"]
Sep 10 00:51:26.785451 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:51:26.785421    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod631da3ce_c11f_492d_9928_ed81d8d5d166.slice/crio-7d5d8a6b882a2c6eed44be6dcbeabfcc91c74e5ee415ae6a29f0c02297bc1ed8 WatchSource:0}: Error finding container 7d5d8a6b882a2c6eed44be6dcbeabfcc91c74e5ee415ae6a29f0c02297bc1ed8: Status 404 returned error can't find the container with id 7d5d8a6b882a2c6eed44be6dcbeabfcc91c74e5ee415ae6a29f0c02297bc1ed8
Sep 10 00:51:27.055717 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:27.055642    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kp4ws"]
Sep 10 00:51:27.079684 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:27.079648    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kp4ws"]
Sep 10 00:51:27.079818 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:27.079777    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kp4ws"
Sep 10 00:51:27.082303 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:27.082271    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-gqm55\""
Sep 10 00:51:27.082415 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:27.082306    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 10 00:51:27.239286 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:27.239253    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6f40c51f-1831-47b7-8b0a-e8dcf78bfed0-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-kp4ws\" (UID: \"6f40c51f-1831-47b7-8b0a-e8dcf78bfed0\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kp4ws"
Sep 10 00:51:27.340479 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:27.340441    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6f40c51f-1831-47b7-8b0a-e8dcf78bfed0-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-kp4ws\" (UID: \"6f40c51f-1831-47b7-8b0a-e8dcf78bfed0\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kp4ws"
Sep 10 00:51:27.342811 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:27.342783    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6f40c51f-1831-47b7-8b0a-e8dcf78bfed0-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-kp4ws\" (UID: \"6f40c51f-1831-47b7-8b0a-e8dcf78bfed0\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kp4ws"
Sep 10 00:51:27.389966 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:27.389936    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kp4ws"
Sep 10 00:51:27.514398 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:27.514368    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-4qhvr" event={"ID":"784a9928-9ed1-4f75-afdb-c42ee2030777","Type":"ContainerStarted","Data":"747f91023c353cc436796f32799a2f6310106a18a74865b025592db104c7910c"}
Sep 10 00:51:27.516157 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:27.516134    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-gzrtx" event={"ID":"b71eb054-e5f2-4f16-b9a3-1253c28778ca","Type":"ContainerStarted","Data":"34c63deaf6951297461714fe78f8ecfbca958507d574b55e07478b70edd5c6e0"}
Sep 10 00:51:27.517234 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:27.517212    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c4ddcd77f-hc9bn" event={"ID":"631da3ce-c11f-492d-9928-ed81d8d5d166","Type":"ContainerStarted","Data":"7d5d8a6b882a2c6eed44be6dcbeabfcc91c74e5ee415ae6a29f0c02297bc1ed8"}
Sep 10 00:51:27.519013 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:27.518970    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kw2lh" event={"ID":"58138b99-90e8-4b0a-89bf-891604fbaca8","Type":"ContainerStarted","Data":"d42abb92a7ade5a3e05966c00e6ac941de6c3d5fcdc43883bf6a86b53aa9de72"}
Sep 10 00:51:27.519092 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:27.519037    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kw2lh" event={"ID":"58138b99-90e8-4b0a-89bf-891604fbaca8","Type":"ContainerStarted","Data":"fdde4a6b2603ccc6c9bf683f5db5adc6fad2c8953658dd8106f1873e47d4da71"}
Sep 10 00:51:27.519092 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:27.519075    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-kw2lh"
Sep 10 00:51:27.530756 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:27.530715    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-4qhvr" podStartSLOduration=33.976269578 podStartE2EDuration="36.530701302s" totalImagesPullingTime="2.554431724s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:50:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:51:27.529421038 +0000 UTC m=+104.026720618" watchObservedRunningTime="2026-09-10 00:51:27.530701302 +0000 UTC m=+104.028000883"
Sep 10 00:51:27.543493 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:27.543452    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-gzrtx" podStartSLOduration=33.945714314 podStartE2EDuration="36.543436179s" totalImagesPullingTime="2.597721865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:50:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:51:27.543220568 +0000 UTC m=+104.040520150" watchObservedRunningTime="2026-09-10 00:51:27.543436179 +0000 UTC m=+104.040735760"
Sep 10 00:51:27.559000 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:27.558952    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-kw2lh" podStartSLOduration=66.103535009 podStartE2EDuration="1m9.558940017s" totalImagesPullingTime="3.455405008s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:50:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:51:27.55846812 +0000 UTC m=+104.055767700" watchObservedRunningTime="2026-09-10 00:51:27.558940017 +0000 UTC m=+104.056239596"
Sep 10 00:51:27.681170 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:27.681099    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kp4ws"]
Sep 10 00:51:27.683591 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:51:27.683558    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f40c51f_1831_47b7_8b0a_e8dcf78bfed0.slice/crio-8af7b40806bea4b904db1f05af525290620a6bdcdf9942f0f4b72c8882a76854 WatchSource:0}: Error finding container 8af7b40806bea4b904db1f05af525290620a6bdcdf9942f0f4b72c8882a76854: Status 404 returned error can't find the container with id 8af7b40806bea4b904db1f05af525290620a6bdcdf9942f0f4b72c8882a76854
Sep 10 00:51:28.527050 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:28.526756    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kp4ws" event={"ID":"6f40c51f-1831-47b7-8b0a-e8dcf78bfed0","Type":"ContainerStarted","Data":"8af7b40806bea4b904db1f05af525290620a6bdcdf9942f0f4b72c8882a76854"}
Sep 10 00:51:29.412417 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:29.412382    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-cvjrx"
Sep 10 00:51:30.870631 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:30.870587    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d794c501-ec2b-419a-aefa-b61f7dedad8e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k9rtv\" (UID: \"d794c501-ec2b-419a-aefa-b61f7dedad8e\") " pod="openshift-insights/insights-runtime-extractor-k9rtv"
Sep 10 00:51:30.873212 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:30.873187    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d794c501-ec2b-419a-aefa-b61f7dedad8e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k9rtv\" (UID: \"d794c501-ec2b-419a-aefa-b61f7dedad8e\") " pod="openshift-insights/insights-runtime-extractor-k9rtv"
Sep 10 00:51:31.094516 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:31.094478    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-k9rtv"
Sep 10 00:51:31.542923 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:31.542883    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c4ddcd77f-hc9bn" event={"ID":"631da3ce-c11f-492d-9928-ed81d8d5d166","Type":"ContainerStarted","Data":"9b7964524732463af459dc8e03ef684ef9603f25575f9ad0d9ea84cc5c20f52b"}
Sep 10 00:51:31.543126 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:31.543098    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c4ddcd77f-hc9bn"
Sep 10 00:51:31.544731 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:31.544663    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kp4ws" event={"ID":"6f40c51f-1831-47b7-8b0a-e8dcf78bfed0","Type":"ContainerStarted","Data":"44a30659b803d45c1e8f027c6857e0a81c435b0bcc2acd64609b6d271129c0b1"}
Sep 10 00:51:31.544927 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:31.544898    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kp4ws"
Sep 10 00:51:31.545122 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:31.545103    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c4ddcd77f-hc9bn"
Sep 10 00:51:31.549084 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:31.549058    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kp4ws"
Sep 10 00:51:31.559481 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:31.559440    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c4ddcd77f-hc9bn" podStartSLOduration=2.984438109 podStartE2EDuration="7.559428805s" totalImagesPullingTime="4.574990696s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:51:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:51:31.558320735 +0000 UTC m=+108.055620313" watchObservedRunningTime="2026-09-10 00:51:31.559428805 +0000 UTC m=+108.056728384"
Sep 10 00:51:31.562760 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:31.562736    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-k9rtv"]
Sep 10 00:51:31.565905 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:51:31.565885    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd794c501_ec2b_419a_aefa_b61f7dedad8e.slice/crio-e6d9fdae34834f75a9919882b7d5603bba8a332f689d370fd927a4187c243760 WatchSource:0}: Error finding container e6d9fdae34834f75a9919882b7d5603bba8a332f689d370fd927a4187c243760: Status 404 returned error can't find the container with id e6d9fdae34834f75a9919882b7d5603bba8a332f689d370fd927a4187c243760
Sep 10 00:51:31.573788 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:31.573741    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kp4ws" podStartSLOduration=0.961413751 podStartE2EDuration="4.573721944s" totalImagesPullingTime="3.612308193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:51:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:51:31.573037777 +0000 UTC m=+108.070337355" watchObservedRunningTime="2026-09-10 00:51:31.573721944 +0000 UTC m=+108.071021527"
Sep 10 00:51:32.111577 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:32.111545    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-7ckhj"]
Sep 10 00:51:32.114694 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:32.114675    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-7ckhj"
Sep 10 00:51:32.117334 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:32.117316    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 10 00:51:32.117428 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:32.117411    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 10 00:51:32.117476 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:32.117420    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-fwvz4\""
Sep 10 00:51:32.118610 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:32.118595    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 10 00:51:32.122817 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:32.122798    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-7ckhj"]
Sep 10 00:51:32.283274 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:32.283247    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnmxq\" (UniqueName: \"kubernetes.io/projected/b9d98f37-c819-4628-b504-00d4ce21a7ab-kube-api-access-hnmxq\") pod \"prometheus-operator-6f4946db56-7ckhj\" (UID: \"b9d98f37-c819-4628-b504-00d4ce21a7ab\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-7ckhj"
Sep 10 00:51:32.283436 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:32.283286    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b9d98f37-c819-4628-b504-00d4ce21a7ab-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-7ckhj\" (UID: \"b9d98f37-c819-4628-b504-00d4ce21a7ab\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-7ckhj"
Sep 10 00:51:32.283436 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:32.283312    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b9d98f37-c819-4628-b504-00d4ce21a7ab-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-7ckhj\" (UID: \"b9d98f37-c819-4628-b504-00d4ce21a7ab\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-7ckhj"
Sep 10 00:51:32.283436 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:32.283393    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b9d98f37-c819-4628-b504-00d4ce21a7ab-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-7ckhj\" (UID: \"b9d98f37-c819-4628-b504-00d4ce21a7ab\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-7ckhj"
Sep 10 00:51:32.384424 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:32.384400    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnmxq\" (UniqueName: \"kubernetes.io/projected/b9d98f37-c819-4628-b504-00d4ce21a7ab-kube-api-access-hnmxq\") pod \"prometheus-operator-6f4946db56-7ckhj\" (UID: \"b9d98f37-c819-4628-b504-00d4ce21a7ab\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-7ckhj"
Sep 10 00:51:32.384535 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:32.384436    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b9d98f37-c819-4628-b504-00d4ce21a7ab-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-7ckhj\" (UID: \"b9d98f37-c819-4628-b504-00d4ce21a7ab\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-7ckhj"
Sep 10 00:51:32.384535 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:32.384462    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b9d98f37-c819-4628-b504-00d4ce21a7ab-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-7ckhj\" (UID: \"b9d98f37-c819-4628-b504-00d4ce21a7ab\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-7ckhj"
Sep 10 00:51:32.384535 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:32.384486    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b9d98f37-c819-4628-b504-00d4ce21a7ab-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-7ckhj\" (UID: \"b9d98f37-c819-4628-b504-00d4ce21a7ab\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-7ckhj"
Sep 10 00:51:32.385115 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:32.385088    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b9d98f37-c819-4628-b504-00d4ce21a7ab-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-7ckhj\" (UID: \"b9d98f37-c819-4628-b504-00d4ce21a7ab\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-7ckhj"
Sep 10 00:51:32.386752 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:32.386680    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b9d98f37-c819-4628-b504-00d4ce21a7ab-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-7ckhj\" (UID: \"b9d98f37-c819-4628-b504-00d4ce21a7ab\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-7ckhj"
Sep 10 00:51:32.386862 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:32.386844    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b9d98f37-c819-4628-b504-00d4ce21a7ab-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-7ckhj\" (UID: \"b9d98f37-c819-4628-b504-00d4ce21a7ab\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-7ckhj"
Sep 10 00:51:32.401947 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:32.401917    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnmxq\" (UniqueName: \"kubernetes.io/projected/b9d98f37-c819-4628-b504-00d4ce21a7ab-kube-api-access-hnmxq\") pod \"prometheus-operator-6f4946db56-7ckhj\" (UID: \"b9d98f37-c819-4628-b504-00d4ce21a7ab\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-7ckhj"
Sep 10 00:51:32.424441 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:32.424417    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-7ckhj"
Sep 10 00:51:32.550657 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:32.550618    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k9rtv" event={"ID":"d794c501-ec2b-419a-aefa-b61f7dedad8e","Type":"ContainerStarted","Data":"d3ce586ea3a28416a967c294578520179ebfcd12d2f39c20c7155991221cb4cf"}
Sep 10 00:51:32.550822 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:32.550678    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k9rtv" event={"ID":"d794c501-ec2b-419a-aefa-b61f7dedad8e","Type":"ContainerStarted","Data":"393f046eb9e8d712eefef404a21c3ce2be0c6586f3661f6a3951e22f9ef385f9"}
Sep 10 00:51:32.550822 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:32.550697    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k9rtv" event={"ID":"d794c501-ec2b-419a-aefa-b61f7dedad8e","Type":"ContainerStarted","Data":"e6d9fdae34834f75a9919882b7d5603bba8a332f689d370fd927a4187c243760"}
Sep 10 00:51:32.693244 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:32.693167    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-7ckhj"]
Sep 10 00:51:32.696264 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:51:32.696235    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9d98f37_c819_4628_b504_00d4ce21a7ab.slice/crio-62d8545709940ee85079a946035619f08857c389d4533fbbe928733ea9baa466 WatchSource:0}: Error finding container 62d8545709940ee85079a946035619f08857c389d4533fbbe928733ea9baa466: Status 404 returned error can't find the container with id 62d8545709940ee85079a946035619f08857c389d4533fbbe928733ea9baa466
Sep 10 00:51:33.505415 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:33.505374    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b9d4975f4-mm9xg"]
Sep 10 00:51:33.509092 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:33.509054    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b9d4975f4-mm9xg"
Sep 10 00:51:33.513049 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:33.513026    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 10 00:51:33.513224 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:33.513037    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 10 00:51:33.513224 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:33.513081    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jlrnr\""
Sep 10 00:51:33.513224 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:33.513090    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 10 00:51:33.513224 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:33.513042    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 10 00:51:33.513853 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:33.513508    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 10 00:51:33.513853 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:33.513735    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 10 00:51:33.514038 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:33.513870    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 10 00:51:33.518540 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:33.518339    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b9d4975f4-mm9xg"]
Sep 10 00:51:33.557159 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:33.557117    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-7ckhj" event={"ID":"b9d98f37-c819-4628-b504-00d4ce21a7ab","Type":"ContainerStarted","Data":"62d8545709940ee85079a946035619f08857c389d4533fbbe928733ea9baa466"}
Sep 10 00:51:33.695870 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:33.695837    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrjlj\" (UniqueName: \"kubernetes.io/projected/f9f9e798-953a-490b-8582-099077dd16fd-kube-api-access-xrjlj\") pod \"console-6b9d4975f4-mm9xg\" (UID: \"f9f9e798-953a-490b-8582-099077dd16fd\") " pod="openshift-console/console-6b9d4975f4-mm9xg"
Sep 10 00:51:33.695870 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:33.695884    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f9f9e798-953a-490b-8582-099077dd16fd-console-serving-cert\") pod \"console-6b9d4975f4-mm9xg\" (UID: \"f9f9e798-953a-490b-8582-099077dd16fd\") " pod="openshift-console/console-6b9d4975f4-mm9xg"
Sep 10 00:51:33.696136 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:33.695932    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f9f9e798-953a-490b-8582-099077dd16fd-service-ca\") pod \"console-6b9d4975f4-mm9xg\" (UID: \"f9f9e798-953a-490b-8582-099077dd16fd\") " pod="openshift-console/console-6b9d4975f4-mm9xg"
Sep 10 00:51:33.696136 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:33.696040    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f9f9e798-953a-490b-8582-099077dd16fd-console-oauth-config\") pod \"console-6b9d4975f4-mm9xg\" (UID: \"f9f9e798-953a-490b-8582-099077dd16fd\") " pod="openshift-console/console-6b9d4975f4-mm9xg"
Sep 10 00:51:33.696136 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:33.696078    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f9f9e798-953a-490b-8582-099077dd16fd-console-config\") pod \"console-6b9d4975f4-mm9xg\" (UID: \"f9f9e798-953a-490b-8582-099077dd16fd\") " pod="openshift-console/console-6b9d4975f4-mm9xg"
Sep 10 00:51:33.696297 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:33.696163    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f9f9e798-953a-490b-8582-099077dd16fd-oauth-serving-cert\") pod \"console-6b9d4975f4-mm9xg\" (UID: \"f9f9e798-953a-490b-8582-099077dd16fd\") " pod="openshift-console/console-6b9d4975f4-mm9xg"
Sep 10 00:51:33.797356 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:33.797266    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrjlj\" (UniqueName: \"kubernetes.io/projected/f9f9e798-953a-490b-8582-099077dd16fd-kube-api-access-xrjlj\") pod \"console-6b9d4975f4-mm9xg\" (UID: \"f9f9e798-953a-490b-8582-099077dd16fd\") " pod="openshift-console/console-6b9d4975f4-mm9xg"
Sep 10 00:51:33.797356 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:33.797320    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f9f9e798-953a-490b-8582-099077dd16fd-console-serving-cert\") pod \"console-6b9d4975f4-mm9xg\" (UID: \"f9f9e798-953a-490b-8582-099077dd16fd\") " pod="openshift-console/console-6b9d4975f4-mm9xg"
Sep 10 00:51:33.797356 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:33.797356    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f9f9e798-953a-490b-8582-099077dd16fd-service-ca\") pod \"console-6b9d4975f4-mm9xg\" (UID: \"f9f9e798-953a-490b-8582-099077dd16fd\") " pod="openshift-console/console-6b9d4975f4-mm9xg"
Sep 10 00:51:33.797618 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:33.797403    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f9f9e798-953a-490b-8582-099077dd16fd-console-oauth-config\") pod \"console-6b9d4975f4-mm9xg\" (UID: \"f9f9e798-953a-490b-8582-099077dd16fd\") " pod="openshift-console/console-6b9d4975f4-mm9xg"
Sep 10 00:51:33.797618 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:33.797428    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f9f9e798-953a-490b-8582-099077dd16fd-console-config\") pod \"console-6b9d4975f4-mm9xg\" (UID: \"f9f9e798-953a-490b-8582-099077dd16fd\") " pod="openshift-console/console-6b9d4975f4-mm9xg"
Sep 10 00:51:33.797618 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:33.797484    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f9f9e798-953a-490b-8582-099077dd16fd-oauth-serving-cert\") pod \"console-6b9d4975f4-mm9xg\" (UID: \"f9f9e798-953a-490b-8582-099077dd16fd\") " pod="openshift-console/console-6b9d4975f4-mm9xg"
Sep 10 00:51:33.798529 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:33.798337    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f9f9e798-953a-490b-8582-099077dd16fd-oauth-serving-cert\") pod \"console-6b9d4975f4-mm9xg\" (UID: \"f9f9e798-953a-490b-8582-099077dd16fd\") " pod="openshift-console/console-6b9d4975f4-mm9xg"
Sep 10 00:51:33.798529 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:33.798353    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f9f9e798-953a-490b-8582-099077dd16fd-service-ca\") pod \"console-6b9d4975f4-mm9xg\" (UID: \"f9f9e798-953a-490b-8582-099077dd16fd\") " pod="openshift-console/console-6b9d4975f4-mm9xg"
Sep 10 00:51:33.798529 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:33.798493    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f9f9e798-953a-490b-8582-099077dd16fd-console-config\") pod \"console-6b9d4975f4-mm9xg\" (UID: \"f9f9e798-953a-490b-8582-099077dd16fd\") " pod="openshift-console/console-6b9d4975f4-mm9xg"
Sep 10 00:51:33.800140 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:33.800118    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f9f9e798-953a-490b-8582-099077dd16fd-console-oauth-config\") pod \"console-6b9d4975f4-mm9xg\" (UID: \"f9f9e798-953a-490b-8582-099077dd16fd\") " pod="openshift-console/console-6b9d4975f4-mm9xg"
Sep 10 00:51:33.800362 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:33.800340    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f9f9e798-953a-490b-8582-099077dd16fd-console-serving-cert\") pod \"console-6b9d4975f4-mm9xg\" (UID: \"f9f9e798-953a-490b-8582-099077dd16fd\") " pod="openshift-console/console-6b9d4975f4-mm9xg"
Sep 10 00:51:33.805802 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:33.805780    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrjlj\" (UniqueName: \"kubernetes.io/projected/f9f9e798-953a-490b-8582-099077dd16fd-kube-api-access-xrjlj\") pod \"console-6b9d4975f4-mm9xg\" (UID: \"f9f9e798-953a-490b-8582-099077dd16fd\") " pod="openshift-console/console-6b9d4975f4-mm9xg"
Sep 10 00:51:33.822552 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:33.822522    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b9d4975f4-mm9xg"
Sep 10 00:51:34.565425 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:34.565390    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k9rtv" event={"ID":"d794c501-ec2b-419a-aefa-b61f7dedad8e","Type":"ContainerStarted","Data":"5e546c6c5413d0422c88568db361019f09c7bcbb16fe541f9abace320858f540"}
Sep 10 00:51:34.567291 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:34.567270    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-7ckhj" event={"ID":"b9d98f37-c819-4628-b504-00d4ce21a7ab","Type":"ContainerStarted","Data":"8b1e54d880bc9369e00c3e92f3194dfa2ba2030f03899448ad2d13e64871945e"}
Sep 10 00:51:34.567348 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:34.567313    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-7ckhj" event={"ID":"b9d98f37-c819-4628-b504-00d4ce21a7ab","Type":"ContainerStarted","Data":"540ba328b061d74d329c7891cb90e519be2489ebc6d0a5f9e790b550bb70a7a0"}
Sep 10 00:51:34.582536 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:34.582448    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-k9rtv" podStartSLOduration=33.922446007 podStartE2EDuration="36.582432714s" totalImagesPullingTime="2.659986707s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 00:50:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:51:34.581551079 +0000 UTC m=+111.078850658" watchObservedRunningTime="2026-09-10 00:51:34.582432714 +0000 UTC m=+111.079732292"
Sep 10 00:51:34.598423 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:34.598395    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b9d4975f4-mm9xg"]
Sep 10 00:51:34.602578 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:51:34.602543    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9f9e798_953a_490b_8582_099077dd16fd.slice/crio-6907ea2781ec4fd0abd519d825f976bc62f27f62f211f9215cfda2307e2b9142 WatchSource:0}: Error finding container 6907ea2781ec4fd0abd519d825f976bc62f27f62f211f9215cfda2307e2b9142: Status 404 returned error can't find the container with id 6907ea2781ec4fd0abd519d825f976bc62f27f62f211f9215cfda2307e2b9142
Sep 10 00:51:35.571921 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:35.571886    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b9d4975f4-mm9xg" event={"ID":"f9f9e798-953a-490b-8582-099077dd16fd","Type":"ContainerStarted","Data":"6907ea2781ec4fd0abd519d825f976bc62f27f62f211f9215cfda2307e2b9142"}
Sep 10 00:51:36.470620 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.470558    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-7ckhj" podStartSLOduration=2.8355935150000002 podStartE2EDuration="4.470538359s" totalImagesPullingTime="1.634944844s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:51:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:51:34.600847733 +0000 UTC m=+111.098147312" watchObservedRunningTime="2026-09-10 00:51:36.470538359 +0000 UTC m=+112.967837940"
Sep 10 00:51:36.471319 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.471296    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-lxhqc"]
Sep 10 00:51:36.474712 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.474689    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-2d77r"]
Sep 10 00:51:36.475032 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.475013    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxhqc"
Sep 10 00:51:36.477924 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.477899    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 10 00:51:36.478062 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.477946    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-j74jr\""
Sep 10 00:51:36.478128 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.478057    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 10 00:51:36.478878 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.478860    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-2d77r"
Sep 10 00:51:36.481056 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.481029    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 10 00:51:36.481467 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.481326    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-vd9qv\""
Sep 10 00:51:36.481467 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.481326    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 10 00:51:36.481467 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.481440    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 10 00:51:36.489270 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.489186    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-2d77r"]
Sep 10 00:51:36.490795 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.490694    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-2zflc"]
Sep 10 00:51:36.494412 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.494390    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-lxhqc"]
Sep 10 00:51:36.494581 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.494566    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2zflc"
Sep 10 00:51:36.497345 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.497175    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 10 00:51:36.497546 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.497525    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 10 00:51:36.497632 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.497556    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 10 00:51:36.497902 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.497883    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-gs6kj\""
Sep 10 00:51:36.516236 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.516207    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5b956a55-24be-4dd1-b187-4fa5c389dc14-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-2d77r\" (UID: \"5b956a55-24be-4dd1-b187-4fa5c389dc14\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2d77r"
Sep 10 00:51:36.516375 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.516292    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5b956a55-24be-4dd1-b187-4fa5c389dc14-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-2d77r\" (UID: \"5b956a55-24be-4dd1-b187-4fa5c389dc14\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2d77r"
Sep 10 00:51:36.516375 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.516332    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/70e3b137-1fc5-47a2-b0d1-86af94a809e5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2zflc\" (UID: \"70e3b137-1fc5-47a2-b0d1-86af94a809e5\") " pod="openshift-monitoring/node-exporter-2zflc"
Sep 10 00:51:36.516375 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.516363    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sc4js\" (UniqueName: \"kubernetes.io/projected/5b956a55-24be-4dd1-b187-4fa5c389dc14-kube-api-access-sc4js\") pod \"kube-state-metrics-74cb57df7-2d77r\" (UID: \"5b956a55-24be-4dd1-b187-4fa5c389dc14\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2d77r"
Sep 10 00:51:36.516532 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.516389    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/70e3b137-1fc5-47a2-b0d1-86af94a809e5-node-exporter-accelerators-collector-config\") pod \"node-exporter-2zflc\" (UID: \"70e3b137-1fc5-47a2-b0d1-86af94a809e5\") " pod="openshift-monitoring/node-exporter-2zflc"
Sep 10 00:51:36.516532 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.516425    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6ca637d2-53c4-424d-8f3b-c6ca6ea03d2c-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-lxhqc\" (UID: \"6ca637d2-53c4-424d-8f3b-c6ca6ea03d2c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxhqc"
Sep 10 00:51:36.516532 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.516462    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/70e3b137-1fc5-47a2-b0d1-86af94a809e5-sys\") pod \"node-exporter-2zflc\" (UID: \"70e3b137-1fc5-47a2-b0d1-86af94a809e5\") " pod="openshift-monitoring/node-exporter-2zflc"
Sep 10 00:51:36.516532 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.516489    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6ca637d2-53c4-424d-8f3b-c6ca6ea03d2c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-lxhqc\" (UID: \"6ca637d2-53c4-424d-8f3b-c6ca6ea03d2c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxhqc"
Sep 10 00:51:36.516532 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.516523    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/70e3b137-1fc5-47a2-b0d1-86af94a809e5-root\") pod \"node-exporter-2zflc\" (UID: \"70e3b137-1fc5-47a2-b0d1-86af94a809e5\") " pod="openshift-monitoring/node-exporter-2zflc"
Sep 10 00:51:36.516763 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.516550    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70e3b137-1fc5-47a2-b0d1-86af94a809e5-metrics-client-ca\") pod \"node-exporter-2zflc\" (UID: \"70e3b137-1fc5-47a2-b0d1-86af94a809e5\") " pod="openshift-monitoring/node-exporter-2zflc"
Sep 10 00:51:36.516763 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.516595    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5b956a55-24be-4dd1-b187-4fa5c389dc14-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-2d77r\" (UID: \"5b956a55-24be-4dd1-b187-4fa5c389dc14\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2d77r"
Sep 10 00:51:36.516763 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.516659    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/70e3b137-1fc5-47a2-b0d1-86af94a809e5-node-exporter-textfile\") pod \"node-exporter-2zflc\" (UID: \"70e3b137-1fc5-47a2-b0d1-86af94a809e5\") " pod="openshift-monitoring/node-exporter-2zflc"
Sep 10 00:51:36.516763 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.516724    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5b956a55-24be-4dd1-b187-4fa5c389dc14-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-2d77r\" (UID: \"5b956a55-24be-4dd1-b187-4fa5c389dc14\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2d77r"
Sep 10 00:51:36.516909 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.516759    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/70e3b137-1fc5-47a2-b0d1-86af94a809e5-node-exporter-tls\") pod \"node-exporter-2zflc\" (UID: \"70e3b137-1fc5-47a2-b0d1-86af94a809e5\") " pod="openshift-monitoring/node-exporter-2zflc"
Sep 10 00:51:36.516909 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.516786    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjvc8\" (UniqueName: \"kubernetes.io/projected/70e3b137-1fc5-47a2-b0d1-86af94a809e5-kube-api-access-hjvc8\") pod \"node-exporter-2zflc\" (UID: \"70e3b137-1fc5-47a2-b0d1-86af94a809e5\") " pod="openshift-monitoring/node-exporter-2zflc"
Sep 10 00:51:36.516909 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.516827    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6ca637d2-53c4-424d-8f3b-c6ca6ea03d2c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-lxhqc\" (UID: \"6ca637d2-53c4-424d-8f3b-c6ca6ea03d2c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxhqc"
Sep 10 00:51:36.516909 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.516870    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpz96\" (UniqueName: \"kubernetes.io/projected/6ca637d2-53c4-424d-8f3b-c6ca6ea03d2c-kube-api-access-jpz96\") pod \"openshift-state-metrics-657fd6c54-lxhqc\" (UID: \"6ca637d2-53c4-424d-8f3b-c6ca6ea03d2c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxhqc"
Sep 10 00:51:36.517057 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.516907    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5b956a55-24be-4dd1-b187-4fa5c389dc14-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-2d77r\" (UID: \"5b956a55-24be-4dd1-b187-4fa5c389dc14\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2d77r"
Sep 10 00:51:36.517057 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.516934    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/70e3b137-1fc5-47a2-b0d1-86af94a809e5-node-exporter-wtmp\") pod \"node-exporter-2zflc\" (UID: \"70e3b137-1fc5-47a2-b0d1-86af94a809e5\") " pod="openshift-monitoring/node-exporter-2zflc"
Sep 10 00:51:36.617706 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.617673    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/70e3b137-1fc5-47a2-b0d1-86af94a809e5-sys\") pod \"node-exporter-2zflc\" (UID: \"70e3b137-1fc5-47a2-b0d1-86af94a809e5\") " pod="openshift-monitoring/node-exporter-2zflc"
Sep 10 00:51:36.619540 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.617714    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6ca637d2-53c4-424d-8f3b-c6ca6ea03d2c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-lxhqc\" (UID: \"6ca637d2-53c4-424d-8f3b-c6ca6ea03d2c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxhqc"
Sep 10 00:51:36.619540 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.617748    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/70e3b137-1fc5-47a2-b0d1-86af94a809e5-root\") pod \"node-exporter-2zflc\" (UID: \"70e3b137-1fc5-47a2-b0d1-86af94a809e5\") " pod="openshift-monitoring/node-exporter-2zflc"
Sep 10 00:51:36.619540 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.617772    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70e3b137-1fc5-47a2-b0d1-86af94a809e5-metrics-client-ca\") pod \"node-exporter-2zflc\" (UID: \"70e3b137-1fc5-47a2-b0d1-86af94a809e5\") " pod="openshift-monitoring/node-exporter-2zflc"
Sep 10 00:51:36.619540 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.617783    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/70e3b137-1fc5-47a2-b0d1-86af94a809e5-sys\") pod \"node-exporter-2zflc\" (UID: \"70e3b137-1fc5-47a2-b0d1-86af94a809e5\") " pod="openshift-monitoring/node-exporter-2zflc"
Sep 10 00:51:36.619540 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.617821    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5b956a55-24be-4dd1-b187-4fa5c389dc14-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-2d77r\" (UID: \"5b956a55-24be-4dd1-b187-4fa5c389dc14\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2d77r"
Sep 10 00:51:36.619540 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.617849    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/70e3b137-1fc5-47a2-b0d1-86af94a809e5-node-exporter-textfile\") pod \"node-exporter-2zflc\" (UID: \"70e3b137-1fc5-47a2-b0d1-86af94a809e5\") " pod="openshift-monitoring/node-exporter-2zflc"
Sep 10 00:51:36.619540 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.617872    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5b956a55-24be-4dd1-b187-4fa5c389dc14-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-2d77r\" (UID: \"5b956a55-24be-4dd1-b187-4fa5c389dc14\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2d77r"
Sep 10 00:51:36.619540 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.618103    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/70e3b137-1fc5-47a2-b0d1-86af94a809e5-node-exporter-tls\") pod \"node-exporter-2zflc\" (UID: \"70e3b137-1fc5-47a2-b0d1-86af94a809e5\") " pod="openshift-monitoring/node-exporter-2zflc"
Sep 10 00:51:36.619540 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.618143    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjvc8\" (UniqueName: \"kubernetes.io/projected/70e3b137-1fc5-47a2-b0d1-86af94a809e5-kube-api-access-hjvc8\") pod \"node-exporter-2zflc\" (UID: \"70e3b137-1fc5-47a2-b0d1-86af94a809e5\") " pod="openshift-monitoring/node-exporter-2zflc"
Sep 10 00:51:36.619540 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.618143    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/70e3b137-1fc5-47a2-b0d1-86af94a809e5-root\") pod \"node-exporter-2zflc\" (UID: \"70e3b137-1fc5-47a2-b0d1-86af94a809e5\") " pod="openshift-monitoring/node-exporter-2zflc"
Sep 10 00:51:36.619540 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.618172    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6ca637d2-53c4-424d-8f3b-c6ca6ea03d2c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-lxhqc\" (UID: \"6ca637d2-53c4-424d-8f3b-c6ca6ea03d2c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxhqc"
Sep 10 00:51:36.619540 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.618229    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpz96\" (UniqueName: \"kubernetes.io/projected/6ca637d2-53c4-424d-8f3b-c6ca6ea03d2c-kube-api-access-jpz96\") pod \"openshift-state-metrics-657fd6c54-lxhqc\" (UID: \"6ca637d2-53c4-424d-8f3b-c6ca6ea03d2c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxhqc"
Sep 10 00:51:36.619540 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:51:36.618259    2563 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 10 00:51:36.619540 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.618301    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5b956a55-24be-4dd1-b187-4fa5c389dc14-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-2d77r\" (UID: \"5b956a55-24be-4dd1-b187-4fa5c389dc14\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2d77r"
Sep 10 00:51:36.619540 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:51:36.618332    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ca637d2-53c4-424d-8f3b-c6ca6ea03d2c-openshift-state-metrics-tls podName:6ca637d2-53c4-424d-8f3b-c6ca6ea03d2c nodeName:}" failed. No retries permitted until 2026-09-10 00:51:37.118309381 +0000 UTC m=+113.615608957 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/6ca637d2-53c4-424d-8f3b-c6ca6ea03d2c-openshift-state-metrics-tls") pod "openshift-state-metrics-657fd6c54-lxhqc" (UID: "6ca637d2-53c4-424d-8f3b-c6ca6ea03d2c") : secret "openshift-state-metrics-tls" not found
Sep 10 00:51:36.619540 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.618348    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/70e3b137-1fc5-47a2-b0d1-86af94a809e5-node-exporter-textfile\") pod \"node-exporter-2zflc\" (UID: \"70e3b137-1fc5-47a2-b0d1-86af94a809e5\") " pod="openshift-monitoring/node-exporter-2zflc"
Sep 10 00:51:36.620357 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.618379    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/70e3b137-1fc5-47a2-b0d1-86af94a809e5-node-exporter-wtmp\") pod \"node-exporter-2zflc\" (UID: \"70e3b137-1fc5-47a2-b0d1-86af94a809e5\") " pod="openshift-monitoring/node-exporter-2zflc"
Sep 10 00:51:36.620357 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.618417    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5b956a55-24be-4dd1-b187-4fa5c389dc14-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-2d77r\" (UID: \"5b956a55-24be-4dd1-b187-4fa5c389dc14\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2d77r"
Sep 10 00:51:36.620357 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.618454    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5b956a55-24be-4dd1-b187-4fa5c389dc14-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-2d77r\" (UID: \"5b956a55-24be-4dd1-b187-4fa5c389dc14\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2d77r"
Sep 10 00:51:36.620357 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.618485    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/70e3b137-1fc5-47a2-b0d1-86af94a809e5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2zflc\" (UID: \"70e3b137-1fc5-47a2-b0d1-86af94a809e5\") " pod="openshift-monitoring/node-exporter-2zflc"
Sep 10 00:51:36.620357 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.618510    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sc4js\" (UniqueName: \"kubernetes.io/projected/5b956a55-24be-4dd1-b187-4fa5c389dc14-kube-api-access-sc4js\") pod \"kube-state-metrics-74cb57df7-2d77r\" (UID: \"5b956a55-24be-4dd1-b187-4fa5c389dc14\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2d77r"
Sep 10 00:51:36.620357 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.618562    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/70e3b137-1fc5-47a2-b0d1-86af94a809e5-node-exporter-accelerators-collector-config\") pod \"node-exporter-2zflc\" (UID: \"70e3b137-1fc5-47a2-b0d1-86af94a809e5\") " pod="openshift-monitoring/node-exporter-2zflc"
Sep 10 00:51:36.620357 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.618595    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6ca637d2-53c4-424d-8f3b-c6ca6ea03d2c-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-lxhqc\" (UID: \"6ca637d2-53c4-424d-8f3b-c6ca6ea03d2c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxhqc"
Sep 10 00:51:36.620357 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.618609    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70e3b137-1fc5-47a2-b0d1-86af94a809e5-metrics-client-ca\") pod \"node-exporter-2zflc\" (UID: \"70e3b137-1fc5-47a2-b0d1-86af94a809e5\") " pod="openshift-monitoring/node-exporter-2zflc"
Sep 10 00:51:36.620357 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.618609    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5b956a55-24be-4dd1-b187-4fa5c389dc14-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-2d77r\" (UID: \"5b956a55-24be-4dd1-b187-4fa5c389dc14\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2d77r"
Sep 10 00:51:36.620357 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.618692    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5b956a55-24be-4dd1-b187-4fa5c389dc14-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-2d77r\" (UID: \"5b956a55-24be-4dd1-b187-4fa5c389dc14\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2d77r"
Sep 10 00:51:36.620357 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:51:36.618708    2563 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 10 00:51:36.620357 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:51:36.618768    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b956a55-24be-4dd1-b187-4fa5c389dc14-kube-state-metrics-tls podName:5b956a55-24be-4dd1-b187-4fa5c389dc14 nodeName:}" failed. No retries permitted until 2026-09-10 00:51:37.118749409 +0000 UTC m=+113.616048981 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/5b956a55-24be-4dd1-b187-4fa5c389dc14-kube-state-metrics-tls") pod "kube-state-metrics-74cb57df7-2d77r" (UID: "5b956a55-24be-4dd1-b187-4fa5c389dc14") : secret "kube-state-metrics-tls" not found
Sep 10 00:51:36.620357 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.618927    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/70e3b137-1fc5-47a2-b0d1-86af94a809e5-node-exporter-wtmp\") pod \"node-exporter-2zflc\" (UID: \"70e3b137-1fc5-47a2-b0d1-86af94a809e5\") " pod="openshift-monitoring/node-exporter-2zflc"
Sep 10 00:51:36.620357 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.619217    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5b956a55-24be-4dd1-b187-4fa5c389dc14-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-2d77r\" (UID: \"5b956a55-24be-4dd1-b187-4fa5c389dc14\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2d77r"
Sep 10 00:51:36.620357 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.619247    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/70e3b137-1fc5-47a2-b0d1-86af94a809e5-node-exporter-accelerators-collector-config\") pod \"node-exporter-2zflc\" (UID: \"70e3b137-1fc5-47a2-b0d1-86af94a809e5\") " pod="openshift-monitoring/node-exporter-2zflc"
Sep 10 00:51:36.620357 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.619325    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6ca637d2-53c4-424d-8f3b-c6ca6ea03d2c-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-lxhqc\" (UID: \"6ca637d2-53c4-424d-8f3b-c6ca6ea03d2c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxhqc"
Sep 10 00:51:36.621476 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.621451    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/70e3b137-1fc5-47a2-b0d1-86af94a809e5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2zflc\" (UID: \"70e3b137-1fc5-47a2-b0d1-86af94a809e5\") " pod="openshift-monitoring/node-exporter-2zflc"
Sep 10 00:51:36.621796 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.621741    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5b956a55-24be-4dd1-b187-4fa5c389dc14-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-2d77r\" (UID: \"5b956a55-24be-4dd1-b187-4fa5c389dc14\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2d77r"
Sep 10 00:51:36.621887 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.621846    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6ca637d2-53c4-424d-8f3b-c6ca6ea03d2c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-lxhqc\" (UID: \"6ca637d2-53c4-424d-8f3b-c6ca6ea03d2c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxhqc"
Sep 10 00:51:36.622163 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.622141    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/70e3b137-1fc5-47a2-b0d1-86af94a809e5-node-exporter-tls\") pod \"node-exporter-2zflc\" (UID: \"70e3b137-1fc5-47a2-b0d1-86af94a809e5\") " pod="openshift-monitoring/node-exporter-2zflc"
Sep 10 00:51:36.627910 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.627866    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpz96\" (UniqueName: \"kubernetes.io/projected/6ca637d2-53c4-424d-8f3b-c6ca6ea03d2c-kube-api-access-jpz96\") pod \"openshift-state-metrics-657fd6c54-lxhqc\" (UID: \"6ca637d2-53c4-424d-8f3b-c6ca6ea03d2c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxhqc"
Sep 10 00:51:36.628066 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.627916    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjvc8\" (UniqueName: \"kubernetes.io/projected/70e3b137-1fc5-47a2-b0d1-86af94a809e5-kube-api-access-hjvc8\") pod \"node-exporter-2zflc\" (UID: \"70e3b137-1fc5-47a2-b0d1-86af94a809e5\") " pod="openshift-monitoring/node-exporter-2zflc"
Sep 10 00:51:36.628234 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.628212    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sc4js\" (UniqueName: \"kubernetes.io/projected/5b956a55-24be-4dd1-b187-4fa5c389dc14-kube-api-access-sc4js\") pod \"kube-state-metrics-74cb57df7-2d77r\" (UID: \"5b956a55-24be-4dd1-b187-4fa5c389dc14\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2d77r"
Sep 10 00:51:36.807630 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:36.807537    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2zflc"
Sep 10 00:51:36.820926 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:51:36.820893    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70e3b137_1fc5_47a2_b0d1_86af94a809e5.slice/crio-de20c629f0c18bb5eeb086ff690bc45f9ca35bb915e62415c178c43ef99de5b8 WatchSource:0}: Error finding container de20c629f0c18bb5eeb086ff690bc45f9ca35bb915e62415c178c43ef99de5b8: Status 404 returned error can't find the container with id de20c629f0c18bb5eeb086ff690bc45f9ca35bb915e62415c178c43ef99de5b8
Sep 10 00:51:37.124062 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:37.124016    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6ca637d2-53c4-424d-8f3b-c6ca6ea03d2c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-lxhqc\" (UID: \"6ca637d2-53c4-424d-8f3b-c6ca6ea03d2c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxhqc"
Sep 10 00:51:37.124263 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:37.124211    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5b956a55-24be-4dd1-b187-4fa5c389dc14-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-2d77r\" (UID: \"5b956a55-24be-4dd1-b187-4fa5c389dc14\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2d77r"
Sep 10 00:51:37.126768 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:37.126745    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6ca637d2-53c4-424d-8f3b-c6ca6ea03d2c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-lxhqc\" (UID: \"6ca637d2-53c4-424d-8f3b-c6ca6ea03d2c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxhqc"
Sep 10 00:51:37.126961 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:37.126939    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5b956a55-24be-4dd1-b187-4fa5c389dc14-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-2d77r\" (UID: \"5b956a55-24be-4dd1-b187-4fa5c389dc14\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-2d77r"
Sep 10 00:51:37.388521 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:37.388432    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxhqc"
Sep 10 00:51:37.400437 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:37.400356    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-2d77r"
Sep 10 00:51:37.529377 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:37.529162    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-kw2lh"
Sep 10 00:51:37.587194 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:37.587158    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2zflc" event={"ID":"70e3b137-1fc5-47a2-b0d1-86af94a809e5","Type":"ContainerStarted","Data":"de20c629f0c18bb5eeb086ff690bc45f9ca35bb915e62415c178c43ef99de5b8"}
Sep 10 00:51:37.693246 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:37.693113    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-lxhqc"]
Sep 10 00:51:37.697281 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:37.697240    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-2d77r"]
Sep 10 00:51:37.700641 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:51:37.700584    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ca637d2_53c4_424d_8f3b_c6ca6ea03d2c.slice/crio-e12a90e9e3ad4d33423e8a3020f2a59afd536751ad03ed164c244e05cb36c962 WatchSource:0}: Error finding container e12a90e9e3ad4d33423e8a3020f2a59afd536751ad03ed164c244e05cb36c962: Status 404 returned error can't find the container with id e12a90e9e3ad4d33423e8a3020f2a59afd536751ad03ed164c244e05cb36c962
Sep 10 00:51:38.592442 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:38.592398    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-2d77r" event={"ID":"5b956a55-24be-4dd1-b187-4fa5c389dc14","Type":"ContainerStarted","Data":"21d8046e9cf698f26af84014dc9f623a625eb4eddca2dc701ada1c00af49b892"}
Sep 10 00:51:38.594141 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:38.594109    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxhqc" event={"ID":"6ca637d2-53c4-424d-8f3b-c6ca6ea03d2c","Type":"ContainerStarted","Data":"ece280f32a9478da59f7d208aac07476fe623d6c6bb112b6182a34678b59da94"}
Sep 10 00:51:38.594301 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:38.594169    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxhqc" event={"ID":"6ca637d2-53c4-424d-8f3b-c6ca6ea03d2c","Type":"ContainerStarted","Data":"e12a90e9e3ad4d33423e8a3020f2a59afd536751ad03ed164c244e05cb36c962"}
Sep 10 00:51:39.599928 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:39.599890    2563 generic.go:356] "Generic (PLEG): container finished" podID="70e3b137-1fc5-47a2-b0d1-86af94a809e5" containerID="c7e35f49d886473ef89d9e5555fc2f1c7561ba1c6c242b651478307d234ed40a" exitCode=0
Sep 10 00:51:39.600395 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:39.600010    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2zflc" event={"ID":"70e3b137-1fc5-47a2-b0d1-86af94a809e5","Type":"ContainerDied","Data":"c7e35f49d886473ef89d9e5555fc2f1c7561ba1c6c242b651478307d234ed40a"}
Sep 10 00:51:39.602179 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:39.602147    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxhqc" event={"ID":"6ca637d2-53c4-424d-8f3b-c6ca6ea03d2c","Type":"ContainerStarted","Data":"09695a101a4fe9b705de3cbc9704c0f94e155304d6ab65d5777217ce9f745517"}
Sep 10 00:51:39.603826 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:39.603803    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b9d4975f4-mm9xg" event={"ID":"f9f9e798-953a-490b-8582-099077dd16fd","Type":"ContainerStarted","Data":"4cea4ae32b15c6079beea9190ef0bb518e085cdc84b68a878434fe8b2cf03204"}
Sep 10 00:51:39.633828 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:39.633782    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b9d4975f4-mm9xg" podStartSLOduration=2.13296855 podStartE2EDuration="6.633764359s" totalImagesPullingTime="4.500795809s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:51:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:51:39.633127428 +0000 UTC m=+116.130427008" watchObservedRunningTime="2026-09-10 00:51:39.633764359 +0000 UTC m=+116.131063940"
Sep 10 00:51:40.615253 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:40.615174    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-2d77r" event={"ID":"5b956a55-24be-4dd1-b187-4fa5c389dc14","Type":"ContainerStarted","Data":"cf40d65317ec76ed30452bb308ca9dcd32ac3c55a169113875b0b0fa211df401"}
Sep 10 00:51:40.615687 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:40.615322    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-2d77r" event={"ID":"5b956a55-24be-4dd1-b187-4fa5c389dc14","Type":"ContainerStarted","Data":"ecf37f5323b371a61ded9af24e0c64fc899e1e2aba345b5a0461d16247d2d23e"}
Sep 10 00:51:40.615687 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:40.615352    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-2d77r" event={"ID":"5b956a55-24be-4dd1-b187-4fa5c389dc14","Type":"ContainerStarted","Data":"6ca6fadcf8e2209a027a5b4a3d4e743212e711272ebbfabc18cd6de5c56a356f"}
Sep 10 00:51:40.617524 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:40.617497    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxhqc" event={"ID":"6ca637d2-53c4-424d-8f3b-c6ca6ea03d2c","Type":"ContainerStarted","Data":"18640a795e269aed0158eb0f29ffd3efa24cc534884b90307a33f85daf238587"}
Sep 10 00:51:40.619872 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:40.619847    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2zflc" event={"ID":"70e3b137-1fc5-47a2-b0d1-86af94a809e5","Type":"ContainerStarted","Data":"e8015b83d793d41786201f31d46a4578cf057dc8ce0a6d80e8a66d3aa025e4e0"}
Sep 10 00:51:40.619979 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:40.619888    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2zflc" event={"ID":"70e3b137-1fc5-47a2-b0d1-86af94a809e5","Type":"ContainerStarted","Data":"6498e533e0be3547e738c09b823e853586680d4c0ab9db8b5b127b776d92d9ee"}
Sep 10 00:51:40.634536 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:40.634492    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-2d77r" podStartSLOduration=2.121234278 podStartE2EDuration="4.63447785s" totalImagesPullingTime="2.513243572s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:51:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:51:40.63347626 +0000 UTC m=+117.130775852" watchObservedRunningTime="2026-09-10 00:51:40.63447785 +0000 UTC m=+117.131777426"
Sep 10 00:51:40.651257 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:40.651209    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-2zflc" podStartSLOduration=2.302048496 podStartE2EDuration="4.651192794s" totalImagesPullingTime="2.280885482s" totalInitContainerRuntime="68.258816ms" isStatefulPod=true podCreationTimestamp="2026-09-10 00:51:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:51:40.64959407 +0000 UTC m=+117.146893639" watchObservedRunningTime="2026-09-10 00:51:40.651192794 +0000 UTC m=+117.148492373"
Sep 10 00:51:40.666018 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:40.665951    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxhqc" podStartSLOduration=3.625833827 podStartE2EDuration="4.665936238s" totalImagesPullingTime="1.040102411s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:51:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:51:40.664494012 +0000 UTC m=+117.161793592" watchObservedRunningTime="2026-09-10 00:51:40.665936238 +0000 UTC m=+117.163235817"
Sep 10 00:51:42.260499 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.260458    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b9d4975f4-mm9xg"]
Sep 10 00:51:42.693036 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.693001    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 00:51:42.697286 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.697258    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.699856 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.699830    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 10 00:51:42.700075 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.700057    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 10 00:51:42.700243 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.700226    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 10 00:51:42.700388 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.700369    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 10 00:51:42.700459 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.700386    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 10 00:51:42.700459 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.700422    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 10 00:51:42.700459 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.700441    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 10 00:51:42.700594 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.700377    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 10 00:51:42.700594 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.700370    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 10 00:51:42.700888 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.700871    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 10 00:51:42.701927 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.701765    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 10 00:51:42.701927 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.701815    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-gpgd7\""
Sep 10 00:51:42.701927 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.701823    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 10 00:51:42.701927 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.701871    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2godtq8b27jq3\""
Sep 10 00:51:42.702178 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.702049    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 10 00:51:42.702345 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.702329    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 10 00:51:42.704919 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.704894    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 10 00:51:42.713536 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.713506    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 00:51:42.780429 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.780397    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.780429 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.780431    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.780669 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.780452    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.780669 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.780510    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/707530fa-8977-4428-b736-7f41881d20ec-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.780669 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.780556    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.780669 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.780583    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-web-config\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.780669 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.780597    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.780669 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.780620    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.780953 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.780681    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/707530fa-8977-4428-b736-7f41881d20ec-config-out\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.780953 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.780726    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.780953 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.780771    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.780953 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.780795    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.780953 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.780819    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.780953 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.780842    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.780953 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.780872    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.780953 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.780900    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.780953 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.780930    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9p9m9\" (UniqueName: \"kubernetes.io/projected/707530fa-8977-4428-b736-7f41881d20ec-kube-api-access-9p9m9\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.780953 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.780950    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.781262 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.780993    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-config\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.781262 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.781015    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/707530fa-8977-4428-b736-7f41881d20ec-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.881840 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.881788    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-config\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.881840 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.881841    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/707530fa-8977-4428-b736-7f41881d20ec-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.882086 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.881907    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.882086 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.881931    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.882086 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.881958    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.882086 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.882007    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/707530fa-8977-4428-b736-7f41881d20ec-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.882086 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.882039    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.882086 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.882065    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-web-config\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.882086 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.882086    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.882400 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.882116    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.882400 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.882145    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/707530fa-8977-4428-b736-7f41881d20ec-config-out\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.882400 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.882175    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.882400 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.882219    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.882400 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.882241    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.882400 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.882263    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.882400 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.882296    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.882400 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.882327    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.882400 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.882358    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.882400 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.882381    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9p9m9\" (UniqueName: \"kubernetes.io/projected/707530fa-8977-4428-b736-7f41881d20ec-kube-api-access-9p9m9\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.882854 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.882404    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.882854 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.882467    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/707530fa-8977-4428-b736-7f41881d20ec-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.883573 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.883223    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.883573 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.883394    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.883740 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.883705    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.884315 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.884288    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.885100 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.885073    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-web-config\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.885450 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.885424    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.885550 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.885529    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.886707 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.886686    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-config\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.887558 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.887536    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.887711 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.887609    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.887846 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.887823    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/707530fa-8977-4428-b736-7f41881d20ec-config-out\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.889343 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.889320    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.889735 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.889709    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.890154 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.890134    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/707530fa-8977-4428-b736-7f41881d20ec-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.890212 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.890156    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.890212 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.890167    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.890758 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.890735    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.891801 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.891777    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:42.895753 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:42.895735    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9p9m9\" (UniqueName: \"kubernetes.io/projected/707530fa-8977-4428-b736-7f41881d20ec-kube-api-access-9p9m9\") pod \"prometheus-k8s-0\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:43.009358 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:43.009262    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:43.303271 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:43.303189    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 00:51:43.306145 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:51:43.306119    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod707530fa_8977_4428_b736_7f41881d20ec.slice/crio-a1f569317c3788207debb30188c34745c0318d478293368d3afc1d12676831f3 WatchSource:0}: Error finding container a1f569317c3788207debb30188c34745c0318d478293368d3afc1d12676831f3: Status 404 returned error can't find the container with id a1f569317c3788207debb30188c34745c0318d478293368d3afc1d12676831f3
Sep 10 00:51:43.633047 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:43.633010    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"707530fa-8977-4428-b736-7f41881d20ec","Type":"ContainerStarted","Data":"a1f569317c3788207debb30188c34745c0318d478293368d3afc1d12676831f3"}
Sep 10 00:51:43.822674 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:43.822636    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b9d4975f4-mm9xg"
Sep 10 00:51:44.507295 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:44.507217    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-b54b88b4b-q42sn"
Sep 10 00:51:44.642975 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:44.642941    2563 generic.go:356] "Generic (PLEG): container finished" podID="707530fa-8977-4428-b736-7f41881d20ec" containerID="800e0dea4b671f50b626e23ba0c553653435fbc3c4397ef014e1f0ce84b48142" exitCode=0
Sep 10 00:51:44.643128 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:44.643030    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"707530fa-8977-4428-b736-7f41881d20ec","Type":"ContainerDied","Data":"800e0dea4b671f50b626e23ba0c553653435fbc3c4397ef014e1f0ce84b48142"}
Sep 10 00:51:48.663414 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:48.663379    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"707530fa-8977-4428-b736-7f41881d20ec","Type":"ContainerStarted","Data":"e40d01213ad66004c11e521003c94dcc6e99b2399c6f0607cda58a51a2089555"}
Sep 10 00:51:48.663908 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:48.663480    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"707530fa-8977-4428-b736-7f41881d20ec","Type":"ContainerStarted","Data":"db3af957e698ce780597fcf77d24d3f3875f6377ebe7e118fee16426a83b0020"}
Sep 10 00:51:49.671182 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:49.671149    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"707530fa-8977-4428-b736-7f41881d20ec","Type":"ContainerStarted","Data":"2d283fc4d6ab716a6dedb56d0be2982fc8c91056e9af387facf4e85d81417334"}
Sep 10 00:51:50.020940 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:50.020902    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-b54b88b4b-q42sn"]
Sep 10 00:51:50.678097 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:50.678063    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"707530fa-8977-4428-b736-7f41881d20ec","Type":"ContainerStarted","Data":"50dde9009ec5ae128e15464153db8d75083cedede59401844e03ccdb626b4443"}
Sep 10 00:51:50.678476 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:50.678130    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"707530fa-8977-4428-b736-7f41881d20ec","Type":"ContainerStarted","Data":"8a26d97e8a3b0c032408178b84047e58dec1186b2b5df21410d58cc0ab6ce19b"}
Sep 10 00:51:50.678476 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:50.678156    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"707530fa-8977-4428-b736-7f41881d20ec","Type":"ContainerStarted","Data":"7d908bda0bb1ea306b0f1f365fce752bf2a546a61301c387c37a54dae9d0bcd6"}
Sep 10 00:51:50.706001 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:50.705914    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.990720946 podStartE2EDuration="8.705893523s" totalImagesPullingTime="5.701825247s" totalInitContainerRuntime="13.34733ms" isStatefulPod=false podCreationTimestamp="2026-09-10 00:51:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:51:50.703874481 +0000 UTC m=+127.201174095" watchObservedRunningTime="2026-09-10 00:51:50.705893523 +0000 UTC m=+127.203193103"
Sep 10 00:51:53.010034 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:53.009961    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:51:53.890365 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:53.890322    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc958bde-e5aa-468a-ae10-0c49f9438168-metrics-certs\") pod \"network-metrics-daemon-8ltcj\" (UID: \"fc958bde-e5aa-468a-ae10-0c49f9438168\") " pod="openshift-multus/network-metrics-daemon-8ltcj"
Sep 10 00:51:53.892711 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:53.892676    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc958bde-e5aa-468a-ae10-0c49f9438168-metrics-certs\") pod \"network-metrics-daemon-8ltcj\" (UID: \"fc958bde-e5aa-468a-ae10-0c49f9438168\") " pod="openshift-multus/network-metrics-daemon-8ltcj"
Sep 10 00:51:54.011196 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:54.011166    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mhb4c\""
Sep 10 00:51:54.019218 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:54.019198    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8ltcj"
Sep 10 00:51:54.287628 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:54.287600    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8ltcj"]
Sep 10 00:51:54.289824 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:51:54.289791    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc958bde_e5aa_468a_ae10_0c49f9438168.slice/crio-0ac522eb3ef23eb2b3dc13238e6cbfa1b8dd219a6b9cbf806bd0acd1830c3f9b WatchSource:0}: Error finding container 0ac522eb3ef23eb2b3dc13238e6cbfa1b8dd219a6b9cbf806bd0acd1830c3f9b: Status 404 returned error can't find the container with id 0ac522eb3ef23eb2b3dc13238e6cbfa1b8dd219a6b9cbf806bd0acd1830c3f9b
Sep 10 00:51:54.699075 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:54.699037    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8ltcj" event={"ID":"fc958bde-e5aa-468a-ae10-0c49f9438168","Type":"ContainerStarted","Data":"0ac522eb3ef23eb2b3dc13238e6cbfa1b8dd219a6b9cbf806bd0acd1830c3f9b"}
Sep 10 00:51:55.704510 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:55.704471    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8ltcj" event={"ID":"fc958bde-e5aa-468a-ae10-0c49f9438168","Type":"ContainerStarted","Data":"f49ff4b0e58654db4cbb71705f021c57e687a9480edc459d1a86505d5822c205"}
Sep 10 00:51:55.704945 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:55.704527    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8ltcj" event={"ID":"fc958bde-e5aa-468a-ae10-0c49f9438168","Type":"ContainerStarted","Data":"304dd8c4a25f70dc3bb096acc76092cb4a71a048af94cca50affc028a323ae31"}
Sep 10 00:51:55.721501 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:55.721452    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-8ltcj" podStartSLOduration=129.676478888 podStartE2EDuration="2m10.721437094s" totalImagesPullingTime="1.044958206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:49:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:51:55.720068042 +0000 UTC m=+132.217367623" watchObservedRunningTime="2026-09-10 00:51:55.721437094 +0000 UTC m=+132.218737073"
Sep 10 00:51:56.286642 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:56.286604    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-765785bbb6-5tjbc"]
Sep 10 00:51:56.290173 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:56.290151    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-765785bbb6-5tjbc"
Sep 10 00:51:56.299515 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:56.299490    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 10 00:51:56.299896 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:56.299875    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-765785bbb6-5tjbc"]
Sep 10 00:51:56.414718 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:56.414680    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-console-serving-cert\") pod \"console-765785bbb6-5tjbc\" (UID: \"c2a99f59-1632-4ec2-a233-bb22ce55f9a6\") " pod="openshift-console/console-765785bbb6-5tjbc"
Sep 10 00:51:56.414935 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:56.414789    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-service-ca\") pod \"console-765785bbb6-5tjbc\" (UID: \"c2a99f59-1632-4ec2-a233-bb22ce55f9a6\") " pod="openshift-console/console-765785bbb6-5tjbc"
Sep 10 00:51:56.414935 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:56.414829    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-trusted-ca-bundle\") pod \"console-765785bbb6-5tjbc\" (UID: \"c2a99f59-1632-4ec2-a233-bb22ce55f9a6\") " pod="openshift-console/console-765785bbb6-5tjbc"
Sep 10 00:51:56.414935 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:56.414899    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-console-config\") pod \"console-765785bbb6-5tjbc\" (UID: \"c2a99f59-1632-4ec2-a233-bb22ce55f9a6\") " pod="openshift-console/console-765785bbb6-5tjbc"
Sep 10 00:51:56.415144 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:56.414934    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mptww\" (UniqueName: \"kubernetes.io/projected/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-kube-api-access-mptww\") pod \"console-765785bbb6-5tjbc\" (UID: \"c2a99f59-1632-4ec2-a233-bb22ce55f9a6\") " pod="openshift-console/console-765785bbb6-5tjbc"
Sep 10 00:51:56.415144 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:56.415026    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-console-oauth-config\") pod \"console-765785bbb6-5tjbc\" (UID: \"c2a99f59-1632-4ec2-a233-bb22ce55f9a6\") " pod="openshift-console/console-765785bbb6-5tjbc"
Sep 10 00:51:56.415144 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:56.415058    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-oauth-serving-cert\") pod \"console-765785bbb6-5tjbc\" (UID: \"c2a99f59-1632-4ec2-a233-bb22ce55f9a6\") " pod="openshift-console/console-765785bbb6-5tjbc"
Sep 10 00:51:56.515895 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:56.515853    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-console-oauth-config\") pod \"console-765785bbb6-5tjbc\" (UID: \"c2a99f59-1632-4ec2-a233-bb22ce55f9a6\") " pod="openshift-console/console-765785bbb6-5tjbc"
Sep 10 00:51:56.515895 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:56.515895    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-oauth-serving-cert\") pod \"console-765785bbb6-5tjbc\" (UID: \"c2a99f59-1632-4ec2-a233-bb22ce55f9a6\") " pod="openshift-console/console-765785bbb6-5tjbc"
Sep 10 00:51:56.516173 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:56.515944    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-console-serving-cert\") pod \"console-765785bbb6-5tjbc\" (UID: \"c2a99f59-1632-4ec2-a233-bb22ce55f9a6\") " pod="openshift-console/console-765785bbb6-5tjbc"
Sep 10 00:51:56.516173 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:56.515974    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-service-ca\") pod \"console-765785bbb6-5tjbc\" (UID: \"c2a99f59-1632-4ec2-a233-bb22ce55f9a6\") " pod="openshift-console/console-765785bbb6-5tjbc"
Sep 10 00:51:56.516173 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:56.516038    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-trusted-ca-bundle\") pod \"console-765785bbb6-5tjbc\" (UID: \"c2a99f59-1632-4ec2-a233-bb22ce55f9a6\") " pod="openshift-console/console-765785bbb6-5tjbc"
Sep 10 00:51:56.516173 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:56.516112    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-console-config\") pod \"console-765785bbb6-5tjbc\" (UID: \"c2a99f59-1632-4ec2-a233-bb22ce55f9a6\") " pod="openshift-console/console-765785bbb6-5tjbc"
Sep 10 00:51:56.516173 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:56.516146    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mptww\" (UniqueName: \"kubernetes.io/projected/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-kube-api-access-mptww\") pod \"console-765785bbb6-5tjbc\" (UID: \"c2a99f59-1632-4ec2-a233-bb22ce55f9a6\") " pod="openshift-console/console-765785bbb6-5tjbc"
Sep 10 00:51:56.516811 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:56.516786    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-service-ca\") pod \"console-765785bbb6-5tjbc\" (UID: \"c2a99f59-1632-4ec2-a233-bb22ce55f9a6\") " pod="openshift-console/console-765785bbb6-5tjbc"
Sep 10 00:51:56.517010 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:56.516847    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-oauth-serving-cert\") pod \"console-765785bbb6-5tjbc\" (UID: \"c2a99f59-1632-4ec2-a233-bb22ce55f9a6\") " pod="openshift-console/console-765785bbb6-5tjbc"
Sep 10 00:51:56.517010 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:56.516890    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-console-config\") pod \"console-765785bbb6-5tjbc\" (UID: \"c2a99f59-1632-4ec2-a233-bb22ce55f9a6\") " pod="openshift-console/console-765785bbb6-5tjbc"
Sep 10 00:51:56.517149 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:56.516920    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-trusted-ca-bundle\") pod \"console-765785bbb6-5tjbc\" (UID: \"c2a99f59-1632-4ec2-a233-bb22ce55f9a6\") " pod="openshift-console/console-765785bbb6-5tjbc"
Sep 10 00:51:56.518329 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:56.518310    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-console-oauth-config\") pod \"console-765785bbb6-5tjbc\" (UID: \"c2a99f59-1632-4ec2-a233-bb22ce55f9a6\") " pod="openshift-console/console-765785bbb6-5tjbc"
Sep 10 00:51:56.518585 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:56.518568    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-console-serving-cert\") pod \"console-765785bbb6-5tjbc\" (UID: \"c2a99f59-1632-4ec2-a233-bb22ce55f9a6\") " pod="openshift-console/console-765785bbb6-5tjbc"
Sep 10 00:51:56.524382 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:56.524360    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mptww\" (UniqueName: \"kubernetes.io/projected/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-kube-api-access-mptww\") pod \"console-765785bbb6-5tjbc\" (UID: \"c2a99f59-1632-4ec2-a233-bb22ce55f9a6\") " pod="openshift-console/console-765785bbb6-5tjbc"
Sep 10 00:51:56.601065 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:56.601024    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-765785bbb6-5tjbc"
Sep 10 00:51:56.872419 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:56.872345    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-765785bbb6-5tjbc"]
Sep 10 00:51:56.875618 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:51:56.875589    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2a99f59_1632_4ec2_a233_bb22ce55f9a6.slice/crio-e11b5f5f850b02547b904cc45169197c417d5b49f74fec4133e822779c407bb7 WatchSource:0}: Error finding container e11b5f5f850b02547b904cc45169197c417d5b49f74fec4133e822779c407bb7: Status 404 returned error can't find the container with id e11b5f5f850b02547b904cc45169197c417d5b49f74fec4133e822779c407bb7
Sep 10 00:51:57.715248 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:57.715205    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-765785bbb6-5tjbc" event={"ID":"c2a99f59-1632-4ec2-a233-bb22ce55f9a6","Type":"ContainerStarted","Data":"df8f7d29ed3a2cfa739db303395832abd4514c3aa9acb5bb20b4b3508c4d90fe"}
Sep 10 00:51:57.715407 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:57.715271    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-765785bbb6-5tjbc" event={"ID":"c2a99f59-1632-4ec2-a233-bb22ce55f9a6","Type":"ContainerStarted","Data":"e11b5f5f850b02547b904cc45169197c417d5b49f74fec4133e822779c407bb7"}
Sep 10 00:51:57.733952 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:51:57.733900    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-765785bbb6-5tjbc" podStartSLOduration=1.733882964 podStartE2EDuration="1.733882964s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:51:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:51:57.733373135 +0000 UTC m=+134.230672714" watchObservedRunningTime="2026-09-10 00:51:57.733882964 +0000 UTC m=+134.231182544"
Sep 10 00:52:03.744697 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:03.744658    2563 generic.go:356] "Generic (PLEG): container finished" podID="501b4642-4933-4505-afe4-4446b75ce2ee" containerID="f53d06aebaf3888dfad70c920fe80324f43d0fa615f69bd72ea1bee5afefc416" exitCode=0
Sep 10 00:52:03.745123 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:03.744713    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-kdm6r" event={"ID":"501b4642-4933-4505-afe4-4446b75ce2ee","Type":"ContainerDied","Data":"f53d06aebaf3888dfad70c920fe80324f43d0fa615f69bd72ea1bee5afefc416"}
Sep 10 00:52:03.745123 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:03.745092    2563 scope.go:117] "RemoveContainer" containerID="f53d06aebaf3888dfad70c920fe80324f43d0fa615f69bd72ea1bee5afefc416"
Sep 10 00:52:04.613547 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:04.613509    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sgpwp_96504ff2-84b9-442b-a77e-254cbe9d8e5a/serve-healthcheck-canary/0.log"
Sep 10 00:52:04.752009 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:04.751955    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-kdm6r" event={"ID":"501b4642-4933-4505-afe4-4446b75ce2ee","Type":"ContainerStarted","Data":"65025a36b29c5a73e208ef5e4def754676bf5f9bc58b24c52fa8656548e4195a"}
Sep 10 00:52:06.602175 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:06.602136    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-765785bbb6-5tjbc"
Sep 10 00:52:06.602644 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:06.602203    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-765785bbb6-5tjbc"
Sep 10 00:52:06.611584 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:06.611559    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-765785bbb6-5tjbc"
Sep 10 00:52:06.763183 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:06.763155    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-765785bbb6-5tjbc"
Sep 10 00:52:07.282362 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:07.282319    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6b9d4975f4-mm9xg" podUID="f9f9e798-953a-490b-8582-099077dd16fd" containerName="console" containerID="cri-o://4cea4ae32b15c6079beea9190ef0bb518e085cdc84b68a878434fe8b2cf03204" gracePeriod=15
Sep 10 00:52:07.539428 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:07.539357    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b9d4975f4-mm9xg"
Sep 10 00:52:07.714829 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:07.714264    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f9f9e798-953a-490b-8582-099077dd16fd-service-ca\") pod \"f9f9e798-953a-490b-8582-099077dd16fd\" (UID: \"f9f9e798-953a-490b-8582-099077dd16fd\") "
Sep 10 00:52:07.714829 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:07.714385    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f9f9e798-953a-490b-8582-099077dd16fd-console-oauth-config\") pod \"f9f9e798-953a-490b-8582-099077dd16fd\" (UID: \"f9f9e798-953a-490b-8582-099077dd16fd\") "
Sep 10 00:52:07.714829 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:07.714432    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f9f9e798-953a-490b-8582-099077dd16fd-oauth-serving-cert\") pod \"f9f9e798-953a-490b-8582-099077dd16fd\" (UID: \"f9f9e798-953a-490b-8582-099077dd16fd\") "
Sep 10 00:52:07.714829 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:07.714458    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f9f9e798-953a-490b-8582-099077dd16fd-console-config\") pod \"f9f9e798-953a-490b-8582-099077dd16fd\" (UID: \"f9f9e798-953a-490b-8582-099077dd16fd\") "
Sep 10 00:52:07.714829 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:07.714500    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xrjlj\" (UniqueName: \"kubernetes.io/projected/f9f9e798-953a-490b-8582-099077dd16fd-kube-api-access-xrjlj\") pod \"f9f9e798-953a-490b-8582-099077dd16fd\" (UID: \"f9f9e798-953a-490b-8582-099077dd16fd\") "
Sep 10 00:52:07.714829 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:07.714530    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f9f9e798-953a-490b-8582-099077dd16fd-console-serving-cert\") pod \"f9f9e798-953a-490b-8582-099077dd16fd\" (UID: \"f9f9e798-953a-490b-8582-099077dd16fd\") "
Sep 10 00:52:07.715584 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:07.715017    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9f9e798-953a-490b-8582-099077dd16fd-console-config" (OuterVolumeSpecName: "console-config") pod "f9f9e798-953a-490b-8582-099077dd16fd" (UID: "f9f9e798-953a-490b-8582-099077dd16fd"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 00:52:07.715584 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:07.715119    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9f9e798-953a-490b-8582-099077dd16fd-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f9f9e798-953a-490b-8582-099077dd16fd" (UID: "f9f9e798-953a-490b-8582-099077dd16fd"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 00:52:07.715584 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:07.715556    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9f9e798-953a-490b-8582-099077dd16fd-service-ca" (OuterVolumeSpecName: "service-ca") pod "f9f9e798-953a-490b-8582-099077dd16fd" (UID: "f9f9e798-953a-490b-8582-099077dd16fd"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 00:52:07.717086 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:07.716970    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9f9e798-953a-490b-8582-099077dd16fd-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f9f9e798-953a-490b-8582-099077dd16fd" (UID: "f9f9e798-953a-490b-8582-099077dd16fd"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 00:52:07.717422 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:07.717404    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9f9e798-953a-490b-8582-099077dd16fd-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f9f9e798-953a-490b-8582-099077dd16fd" (UID: "f9f9e798-953a-490b-8582-099077dd16fd"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 00:52:07.717515 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:07.717488    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9f9e798-953a-490b-8582-099077dd16fd-kube-api-access-xrjlj" (OuterVolumeSpecName: "kube-api-access-xrjlj") pod "f9f9e798-953a-490b-8582-099077dd16fd" (UID: "f9f9e798-953a-490b-8582-099077dd16fd"). InnerVolumeSpecName "kube-api-access-xrjlj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 00:52:07.766082 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:07.766034    2563 generic.go:356] "Generic (PLEG): container finished" podID="f9f9e798-953a-490b-8582-099077dd16fd" containerID="4cea4ae32b15c6079beea9190ef0bb518e085cdc84b68a878434fe8b2cf03204" exitCode=0
Sep 10 00:52:07.766285 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:07.766085    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b9d4975f4-mm9xg"
Sep 10 00:52:07.766285 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:07.766127    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b9d4975f4-mm9xg" event={"ID":"f9f9e798-953a-490b-8582-099077dd16fd","Type":"ContainerDied","Data":"4cea4ae32b15c6079beea9190ef0bb518e085cdc84b68a878434fe8b2cf03204"}
Sep 10 00:52:07.766285 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:07.766187    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b9d4975f4-mm9xg" event={"ID":"f9f9e798-953a-490b-8582-099077dd16fd","Type":"ContainerDied","Data":"6907ea2781ec4fd0abd519d825f976bc62f27f62f211f9215cfda2307e2b9142"}
Sep 10 00:52:07.766285 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:07.766216    2563 scope.go:117] "RemoveContainer" containerID="4cea4ae32b15c6079beea9190ef0bb518e085cdc84b68a878434fe8b2cf03204"
Sep 10 00:52:07.776436 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:07.776416    2563 scope.go:117] "RemoveContainer" containerID="4cea4ae32b15c6079beea9190ef0bb518e085cdc84b68a878434fe8b2cf03204"
Sep 10 00:52:07.776782 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:52:07.776763    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4cea4ae32b15c6079beea9190ef0bb518e085cdc84b68a878434fe8b2cf03204\": container with ID starting with 4cea4ae32b15c6079beea9190ef0bb518e085cdc84b68a878434fe8b2cf03204 not found: ID does not exist" containerID="4cea4ae32b15c6079beea9190ef0bb518e085cdc84b68a878434fe8b2cf03204"
Sep 10 00:52:07.776826 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:07.776791    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4cea4ae32b15c6079beea9190ef0bb518e085cdc84b68a878434fe8b2cf03204"} err="failed to get container status \"4cea4ae32b15c6079beea9190ef0bb518e085cdc84b68a878434fe8b2cf03204\": rpc error: code = NotFound desc = could not find container \"4cea4ae32b15c6079beea9190ef0bb518e085cdc84b68a878434fe8b2cf03204\": container with ID starting with 4cea4ae32b15c6079beea9190ef0bb518e085cdc84b68a878434fe8b2cf03204 not found: ID does not exist"
Sep 10 00:52:07.807529 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:07.807430    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b9d4975f4-mm9xg"]
Sep 10 00:52:07.813299 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:07.813266    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b9d4975f4-mm9xg"]
Sep 10 00:52:07.816104 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:07.816075    2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f9f9e798-953a-490b-8582-099077dd16fd-console-oauth-config\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:52:07.816104 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:07.816101    2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f9f9e798-953a-490b-8582-099077dd16fd-oauth-serving-cert\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:52:07.816263 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:07.816114    2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f9f9e798-953a-490b-8582-099077dd16fd-console-config\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:52:07.816263 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:07.816126    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xrjlj\" (UniqueName: \"kubernetes.io/projected/f9f9e798-953a-490b-8582-099077dd16fd-kube-api-access-xrjlj\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:52:07.816263 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:07.816136    2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f9f9e798-953a-490b-8582-099077dd16fd-console-serving-cert\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:52:07.816263 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:07.816144    2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f9f9e798-953a-490b-8582-099077dd16fd-service-ca\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:52:08.081368 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:08.081272    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9f9e798-953a-490b-8582-099077dd16fd" path="/var/lib/kubelet/pods/f9f9e798-953a-490b-8582-099077dd16fd/volumes"
Sep 10 00:52:15.040648 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:15.040594    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-b54b88b4b-q42sn" podUID="8cf5ecf8-4fd5-4c6e-99ea-4015ff995450" containerName="registry" containerID="cri-o://bd382025971652e50667c2aee38fe063dcb042c3359bc8940e3d6818659bbbe1" gracePeriod=30
Sep 10 00:52:15.324595 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:15.324562    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b54b88b4b-q42sn"
Sep 10 00:52:15.383718 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:15.383686    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-installation-pull-secrets\") pod \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") "
Sep 10 00:52:15.383900 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:15.383735    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-bound-sa-token\") pod \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") "
Sep 10 00:52:15.383900 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:15.383878    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-registry-tls\") pod \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") "
Sep 10 00:52:15.384054 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:15.383946    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-ca-trust-extracted\") pod \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") "
Sep 10 00:52:15.384054 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:15.383966    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dd5tw\" (UniqueName: \"kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-kube-api-access-dd5tw\") pod \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") "
Sep 10 00:52:15.384054 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:15.384009    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-trusted-ca\") pod \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") "
Sep 10 00:52:15.384054 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:15.384028    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-registry-certificates\") pod \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") "
Sep 10 00:52:15.384054 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:15.384047    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-image-registry-private-configuration\") pod \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\" (UID: \"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450\") "
Sep 10 00:52:15.385566 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:15.385512    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "8cf5ecf8-4fd5-4c6e-99ea-4015ff995450" (UID: "8cf5ecf8-4fd5-4c6e-99ea-4015ff995450"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 00:52:15.385944 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:15.385907    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "8cf5ecf8-4fd5-4c6e-99ea-4015ff995450" (UID: "8cf5ecf8-4fd5-4c6e-99ea-4015ff995450"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 00:52:15.389769 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:15.389634    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "8cf5ecf8-4fd5-4c6e-99ea-4015ff995450" (UID: "8cf5ecf8-4fd5-4c6e-99ea-4015ff995450"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 00:52:15.389769 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:15.389673    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-kube-api-access-dd5tw" (OuterVolumeSpecName: "kube-api-access-dd5tw") pod "8cf5ecf8-4fd5-4c6e-99ea-4015ff995450" (UID: "8cf5ecf8-4fd5-4c6e-99ea-4015ff995450"). InnerVolumeSpecName "kube-api-access-dd5tw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 00:52:15.389972 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:15.389823    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "8cf5ecf8-4fd5-4c6e-99ea-4015ff995450" (UID: "8cf5ecf8-4fd5-4c6e-99ea-4015ff995450"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 00:52:15.389972 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:15.389827    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "8cf5ecf8-4fd5-4c6e-99ea-4015ff995450" (UID: "8cf5ecf8-4fd5-4c6e-99ea-4015ff995450"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 00:52:15.390535 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:15.390509    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "8cf5ecf8-4fd5-4c6e-99ea-4015ff995450" (UID: "8cf5ecf8-4fd5-4c6e-99ea-4015ff995450"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 00:52:15.397073 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:15.397038    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "8cf5ecf8-4fd5-4c6e-99ea-4015ff995450" (UID: "8cf5ecf8-4fd5-4c6e-99ea-4015ff995450"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 00:52:15.485157 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:15.485122    2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-installation-pull-secrets\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:52:15.485321 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:15.485174    2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-bound-sa-token\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:52:15.485321 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:15.485190    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-registry-tls\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:52:15.485321 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:15.485199    2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-ca-trust-extracted\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:52:15.485321 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:15.485208    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dd5tw\" (UniqueName: \"kubernetes.io/projected/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-kube-api-access-dd5tw\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:52:15.485321 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:15.485216    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-trusted-ca\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:52:15.485321 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:15.485224    2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-registry-certificates\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:52:15.485321 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:15.485233    2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450-image-registry-private-configuration\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:52:15.802534 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:15.802490    2563 generic.go:356] "Generic (PLEG): container finished" podID="8cf5ecf8-4fd5-4c6e-99ea-4015ff995450" containerID="bd382025971652e50667c2aee38fe063dcb042c3359bc8940e3d6818659bbbe1" exitCode=0
Sep 10 00:52:15.802710 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:15.802594    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b54b88b4b-q42sn"
Sep 10 00:52:15.802710 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:15.802614    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b54b88b4b-q42sn" event={"ID":"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450","Type":"ContainerDied","Data":"bd382025971652e50667c2aee38fe063dcb042c3359bc8940e3d6818659bbbe1"}
Sep 10 00:52:15.802710 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:15.802705    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b54b88b4b-q42sn" event={"ID":"8cf5ecf8-4fd5-4c6e-99ea-4015ff995450","Type":"ContainerDied","Data":"fd5a45bc81e6eb5906e7e116a8b631f97b081d3ed832432eca75c006dea96a76"}
Sep 10 00:52:15.802827 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:15.802734    2563 scope.go:117] "RemoveContainer" containerID="bd382025971652e50667c2aee38fe063dcb042c3359bc8940e3d6818659bbbe1"
Sep 10 00:52:15.812218 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:15.812198    2563 scope.go:117] "RemoveContainer" containerID="bd382025971652e50667c2aee38fe063dcb042c3359bc8940e3d6818659bbbe1"
Sep 10 00:52:15.812493 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:52:15.812471    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd382025971652e50667c2aee38fe063dcb042c3359bc8940e3d6818659bbbe1\": container with ID starting with bd382025971652e50667c2aee38fe063dcb042c3359bc8940e3d6818659bbbe1 not found: ID does not exist" containerID="bd382025971652e50667c2aee38fe063dcb042c3359bc8940e3d6818659bbbe1"
Sep 10 00:52:15.812556 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:15.812502    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd382025971652e50667c2aee38fe063dcb042c3359bc8940e3d6818659bbbe1"} err="failed to get container status \"bd382025971652e50667c2aee38fe063dcb042c3359bc8940e3d6818659bbbe1\": rpc error: code = NotFound desc = could not find container \"bd382025971652e50667c2aee38fe063dcb042c3359bc8940e3d6818659bbbe1\": container with ID starting with bd382025971652e50667c2aee38fe063dcb042c3359bc8940e3d6818659bbbe1 not found: ID does not exist"
Sep 10 00:52:15.824529 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:15.824497    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-b54b88b4b-q42sn"]
Sep 10 00:52:15.832649 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:15.832615    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-b54b88b4b-q42sn"]
Sep 10 00:52:16.081771 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:16.081696    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8cf5ecf8-4fd5-4c6e-99ea-4015ff995450" path="/var/lib/kubelet/pods/8cf5ecf8-4fd5-4c6e-99ea-4015ff995450/volumes"
Sep 10 00:52:28.860568 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:28.860532    2563 generic.go:356] "Generic (PLEG): container finished" podID="6ebec561-8cb9-40af-92f8-dae61b52833b" containerID="0dd2bbf678e5b232cf5c9db69a7c8b48611750e139533938c785698f4fb29015" exitCode=0
Sep 10 00:52:28.860970 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:28.860607    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rpj5t" event={"ID":"6ebec561-8cb9-40af-92f8-dae61b52833b","Type":"ContainerDied","Data":"0dd2bbf678e5b232cf5c9db69a7c8b48611750e139533938c785698f4fb29015"}
Sep 10 00:52:28.860970 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:28.860955    2563 scope.go:117] "RemoveContainer" containerID="0dd2bbf678e5b232cf5c9db69a7c8b48611750e139533938c785698f4fb29015"
Sep 10 00:52:29.866485 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:29.866446    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rpj5t" event={"ID":"6ebec561-8cb9-40af-92f8-dae61b52833b","Type":"ContainerStarted","Data":"67bab436cb58373623ed598774c97456ccfee99e0d655cb9287ee3aef3a849dd"}
Sep 10 00:52:43.010075 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:43.009963    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:52:43.028490 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:43.028461    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:52:43.941265 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:52:43.941239    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:01.026189 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.026136    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 00:53:01.026734 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.026470    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="707530fa-8977-4428-b736-7f41881d20ec" containerName="prometheus" containerID="cri-o://db3af957e698ce780597fcf77d24d3f3875f6377ebe7e118fee16426a83b0020" gracePeriod=600
Sep 10 00:53:01.026734 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.026515    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="707530fa-8977-4428-b736-7f41881d20ec" containerName="kube-rbac-proxy" containerID="cri-o://8a26d97e8a3b0c032408178b84047e58dec1186b2b5df21410d58cc0ab6ce19b" gracePeriod=600
Sep 10 00:53:01.026734 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.026543    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="707530fa-8977-4428-b736-7f41881d20ec" containerName="kube-rbac-proxy-thanos" containerID="cri-o://50dde9009ec5ae128e15464153db8d75083cedede59401844e03ccdb626b4443" gracePeriod=600
Sep 10 00:53:01.026734 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.026562    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="707530fa-8977-4428-b736-7f41881d20ec" containerName="thanos-sidecar" containerID="cri-o://2d283fc4d6ab716a6dedb56d0be2982fc8c91056e9af387facf4e85d81417334" gracePeriod=600
Sep 10 00:53:01.026734 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.026593    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="707530fa-8977-4428-b736-7f41881d20ec" containerName="kube-rbac-proxy-web" containerID="cri-o://7d908bda0bb1ea306b0f1f365fce752bf2a546a61301c387c37a54dae9d0bcd6" gracePeriod=600
Sep 10 00:53:01.026734 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.026545    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="707530fa-8977-4428-b736-7f41881d20ec" containerName="config-reloader" containerID="cri-o://e40d01213ad66004c11e521003c94dcc6e99b2399c6f0607cda58a51a2089555" gracePeriod=600
Sep 10 00:53:01.271232 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.271197    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:01.283830 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.283752    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-thanos-prometheus-http-client-file\") pod \"707530fa-8977-4428-b736-7f41881d20ec\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") "
Sep 10 00:53:01.283963 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.283845    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-prometheus-k8s-rulefiles-1\") pod \"707530fa-8977-4428-b736-7f41881d20ec\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") "
Sep 10 00:53:01.284068 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.284016    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-prometheus-k8s-rulefiles-2\") pod \"707530fa-8977-4428-b736-7f41881d20ec\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") "
Sep 10 00:53:01.284068 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.284057    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-configmap-metrics-client-ca\") pod \"707530fa-8977-4428-b736-7f41881d20ec\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") "
Sep 10 00:53:01.284183 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.284085    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/707530fa-8977-4428-b736-7f41881d20ec-config-out\") pod \"707530fa-8977-4428-b736-7f41881d20ec\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") "
Sep 10 00:53:01.284183 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.284113    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-prometheus-k8s-rulefiles-0\") pod \"707530fa-8977-4428-b736-7f41881d20ec\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") "
Sep 10 00:53:01.284183 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.284135    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "707530fa-8977-4428-b736-7f41881d20ec" (UID: "707530fa-8977-4428-b736-7f41881d20ec"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 00:53:01.284183 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.284168    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-prometheus-trusted-ca-bundle\") pod \"707530fa-8977-4428-b736-7f41881d20ec\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") "
Sep 10 00:53:01.284371 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.284200    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9p9m9\" (UniqueName: \"kubernetes.io/projected/707530fa-8977-4428-b736-7f41881d20ec-kube-api-access-9p9m9\") pod \"707530fa-8977-4428-b736-7f41881d20ec\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") "
Sep 10 00:53:01.284371 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.284228    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-secret-kube-rbac-proxy\") pod \"707530fa-8977-4428-b736-7f41881d20ec\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") "
Sep 10 00:53:01.284371 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.284277    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"707530fa-8977-4428-b736-7f41881d20ec\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") "
Sep 10 00:53:01.284371 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.284303    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-configmap-serving-certs-ca-bundle\") pod \"707530fa-8977-4428-b736-7f41881d20ec\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") "
Sep 10 00:53:01.284560 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.284364    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "707530fa-8977-4428-b736-7f41881d20ec" (UID: "707530fa-8977-4428-b736-7f41881d20ec"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 00:53:01.285308 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.284628    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:53:01.285308 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.284653    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:53:01.285308 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.284711    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "707530fa-8977-4428-b736-7f41881d20ec" (UID: "707530fa-8977-4428-b736-7f41881d20ec"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 00:53:01.285308 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.284916    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "707530fa-8977-4428-b736-7f41881d20ec" (UID: "707530fa-8977-4428-b736-7f41881d20ec"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 00:53:01.285308 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.285122    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "707530fa-8977-4428-b736-7f41881d20ec" (UID: "707530fa-8977-4428-b736-7f41881d20ec"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 00:53:01.285774 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.285752    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "707530fa-8977-4428-b736-7f41881d20ec" (UID: "707530fa-8977-4428-b736-7f41881d20ec"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 00:53:01.286656 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.286622    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "707530fa-8977-4428-b736-7f41881d20ec" (UID: "707530fa-8977-4428-b736-7f41881d20ec"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 00:53:01.286656 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.286637    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/707530fa-8977-4428-b736-7f41881d20ec-config-out" (OuterVolumeSpecName: "config-out") pod "707530fa-8977-4428-b736-7f41881d20ec" (UID: "707530fa-8977-4428-b736-7f41881d20ec"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 00:53:01.286974 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.286951    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "707530fa-8977-4428-b736-7f41881d20ec" (UID: "707530fa-8977-4428-b736-7f41881d20ec"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 00:53:01.287138 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.287006    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/707530fa-8977-4428-b736-7f41881d20ec-kube-api-access-9p9m9" (OuterVolumeSpecName: "kube-api-access-9p9m9") pod "707530fa-8977-4428-b736-7f41881d20ec" (UID: "707530fa-8977-4428-b736-7f41881d20ec"). InnerVolumeSpecName "kube-api-access-9p9m9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 00:53:01.287349 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.287332    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "707530fa-8977-4428-b736-7f41881d20ec" (UID: "707530fa-8977-4428-b736-7f41881d20ec"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 00:53:01.384907 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.384869    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-config\") pod \"707530fa-8977-4428-b736-7f41881d20ec\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") "
Sep 10 00:53:01.384907 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.384905    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/707530fa-8977-4428-b736-7f41881d20ec-tls-assets\") pod \"707530fa-8977-4428-b736-7f41881d20ec\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") "
Sep 10 00:53:01.385153 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.384924    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-secret-metrics-client-certs\") pod \"707530fa-8977-4428-b736-7f41881d20ec\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") "
Sep 10 00:53:01.385153 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.384954    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-secret-prometheus-k8s-tls\") pod \"707530fa-8977-4428-b736-7f41881d20ec\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") "
Sep 10 00:53:01.385153 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.384996    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/707530fa-8977-4428-b736-7f41881d20ec-prometheus-k8s-db\") pod \"707530fa-8977-4428-b736-7f41881d20ec\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") "
Sep 10 00:53:01.385153 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.385027    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"707530fa-8977-4428-b736-7f41881d20ec\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") "
Sep 10 00:53:01.385153 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.385068    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-configmap-kubelet-serving-ca-bundle\") pod \"707530fa-8977-4428-b736-7f41881d20ec\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") "
Sep 10 00:53:01.385415 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.385177    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-web-config\") pod \"707530fa-8977-4428-b736-7f41881d20ec\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") "
Sep 10 00:53:01.385556 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.385497    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "707530fa-8977-4428-b736-7f41881d20ec" (UID: "707530fa-8977-4428-b736-7f41881d20ec"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 00:53:01.386255 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.385965    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-secret-grpc-tls\") pod \"707530fa-8977-4428-b736-7f41881d20ec\" (UID: \"707530fa-8977-4428-b736-7f41881d20ec\") "
Sep 10 00:53:01.386255 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.386012    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/707530fa-8977-4428-b736-7f41881d20ec-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "707530fa-8977-4428-b736-7f41881d20ec" (UID: "707530fa-8977-4428-b736-7f41881d20ec"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 00:53:01.386804 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.386598    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-prometheus-trusted-ca-bundle\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:53:01.386804 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.386625    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9p9m9\" (UniqueName: \"kubernetes.io/projected/707530fa-8977-4428-b736-7f41881d20ec-kube-api-access-9p9m9\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:53:01.386804 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.386641    2563 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-secret-kube-rbac-proxy\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:53:01.386804 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.386656    2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:53:01.386804 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.386670    2563 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:53:01.386804 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.386684    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/707530fa-8977-4428-b736-7f41881d20ec-prometheus-k8s-db\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:53:01.386804 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.386698    2563 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:53:01.386804 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.386715    2563 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-thanos-prometheus-http-client-file\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:53:01.386804 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.386730    2563 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-configmap-metrics-client-ca\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:53:01.386804 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.386744    2563 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/707530fa-8977-4428-b736-7f41881d20ec-config-out\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:53:01.386804 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.386758    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/707530fa-8977-4428-b736-7f41881d20ec-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:53:01.387542 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.387490    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/707530fa-8977-4428-b736-7f41881d20ec-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "707530fa-8977-4428-b736-7f41881d20ec" (UID: "707530fa-8977-4428-b736-7f41881d20ec"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 00:53:01.387608 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.387557    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-config" (OuterVolumeSpecName: "config") pod "707530fa-8977-4428-b736-7f41881d20ec" (UID: "707530fa-8977-4428-b736-7f41881d20ec"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 00:53:01.387608 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.387571    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "707530fa-8977-4428-b736-7f41881d20ec" (UID: "707530fa-8977-4428-b736-7f41881d20ec"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 00:53:01.387678 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.387595    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "707530fa-8977-4428-b736-7f41881d20ec" (UID: "707530fa-8977-4428-b736-7f41881d20ec"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 00:53:01.387891 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.387872    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "707530fa-8977-4428-b736-7f41881d20ec" (UID: "707530fa-8977-4428-b736-7f41881d20ec"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 00:53:01.388213 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.388197    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "707530fa-8977-4428-b736-7f41881d20ec" (UID: "707530fa-8977-4428-b736-7f41881d20ec"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 00:53:01.397901 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.397881    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-web-config" (OuterVolumeSpecName: "web-config") pod "707530fa-8977-4428-b736-7f41881d20ec" (UID: "707530fa-8977-4428-b736-7f41881d20ec"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 00:53:01.487513 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.487475    2563 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-web-config\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:53:01.487513 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.487507    2563 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-secret-grpc-tls\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:53:01.487513 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.487518    2563 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-config\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:53:01.487745 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.487528    2563 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/707530fa-8977-4428-b736-7f41881d20ec-tls-assets\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:53:01.487745 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.487540    2563 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-secret-metrics-client-certs\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:53:01.487745 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.487554    2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-secret-prometheus-k8s-tls\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:53:01.487745 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:01.487566    2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/707530fa-8977-4428-b736-7f41881d20ec-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:53:02.003439 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.003406    2563 generic.go:356] "Generic (PLEG): container finished" podID="707530fa-8977-4428-b736-7f41881d20ec" containerID="50dde9009ec5ae128e15464153db8d75083cedede59401844e03ccdb626b4443" exitCode=0
Sep 10 00:53:02.003439 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.003434    2563 generic.go:356] "Generic (PLEG): container finished" podID="707530fa-8977-4428-b736-7f41881d20ec" containerID="8a26d97e8a3b0c032408178b84047e58dec1186b2b5df21410d58cc0ab6ce19b" exitCode=0
Sep 10 00:53:02.003439 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.003441    2563 generic.go:356] "Generic (PLEG): container finished" podID="707530fa-8977-4428-b736-7f41881d20ec" containerID="7d908bda0bb1ea306b0f1f365fce752bf2a546a61301c387c37a54dae9d0bcd6" exitCode=0
Sep 10 00:53:02.003439 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.003448    2563 generic.go:356] "Generic (PLEG): container finished" podID="707530fa-8977-4428-b736-7f41881d20ec" containerID="2d283fc4d6ab716a6dedb56d0be2982fc8c91056e9af387facf4e85d81417334" exitCode=0
Sep 10 00:53:02.003808 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.003453    2563 generic.go:356] "Generic (PLEG): container finished" podID="707530fa-8977-4428-b736-7f41881d20ec" containerID="e40d01213ad66004c11e521003c94dcc6e99b2399c6f0607cda58a51a2089555" exitCode=0
Sep 10 00:53:02.003808 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.003458    2563 generic.go:356] "Generic (PLEG): container finished" podID="707530fa-8977-4428-b736-7f41881d20ec" containerID="db3af957e698ce780597fcf77d24d3f3875f6377ebe7e118fee16426a83b0020" exitCode=0
Sep 10 00:53:02.003808 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.003489    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.003808 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.003488    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"707530fa-8977-4428-b736-7f41881d20ec","Type":"ContainerDied","Data":"50dde9009ec5ae128e15464153db8d75083cedede59401844e03ccdb626b4443"}
Sep 10 00:53:02.003808 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.003604    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"707530fa-8977-4428-b736-7f41881d20ec","Type":"ContainerDied","Data":"8a26d97e8a3b0c032408178b84047e58dec1186b2b5df21410d58cc0ab6ce19b"}
Sep 10 00:53:02.003808 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.003647    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"707530fa-8977-4428-b736-7f41881d20ec","Type":"ContainerDied","Data":"7d908bda0bb1ea306b0f1f365fce752bf2a546a61301c387c37a54dae9d0bcd6"}
Sep 10 00:53:02.003808 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.003691    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"707530fa-8977-4428-b736-7f41881d20ec","Type":"ContainerDied","Data":"2d283fc4d6ab716a6dedb56d0be2982fc8c91056e9af387facf4e85d81417334"}
Sep 10 00:53:02.003808 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.003672    2563 scope.go:117] "RemoveContainer" containerID="50dde9009ec5ae128e15464153db8d75083cedede59401844e03ccdb626b4443"
Sep 10 00:53:02.003808 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.003733    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"707530fa-8977-4428-b736-7f41881d20ec","Type":"ContainerDied","Data":"e40d01213ad66004c11e521003c94dcc6e99b2399c6f0607cda58a51a2089555"}
Sep 10 00:53:02.003808 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.003774    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"707530fa-8977-4428-b736-7f41881d20ec","Type":"ContainerDied","Data":"db3af957e698ce780597fcf77d24d3f3875f6377ebe7e118fee16426a83b0020"}
Sep 10 00:53:02.003808 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.003814    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"707530fa-8977-4428-b736-7f41881d20ec","Type":"ContainerDied","Data":"a1f569317c3788207debb30188c34745c0318d478293368d3afc1d12676831f3"}
Sep 10 00:53:02.012141 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.012026    2563 scope.go:117] "RemoveContainer" containerID="8a26d97e8a3b0c032408178b84047e58dec1186b2b5df21410d58cc0ab6ce19b"
Sep 10 00:53:02.019131 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.019115    2563 scope.go:117] "RemoveContainer" containerID="7d908bda0bb1ea306b0f1f365fce752bf2a546a61301c387c37a54dae9d0bcd6"
Sep 10 00:53:02.025659 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.025642    2563 scope.go:117] "RemoveContainer" containerID="2d283fc4d6ab716a6dedb56d0be2982fc8c91056e9af387facf4e85d81417334"
Sep 10 00:53:02.028877 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.028853    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 00:53:02.033866 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.033757    2563 scope.go:117] "RemoveContainer" containerID="e40d01213ad66004c11e521003c94dcc6e99b2399c6f0607cda58a51a2089555"
Sep 10 00:53:02.034391 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.034374    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 00:53:02.040396 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.040377    2563 scope.go:117] "RemoveContainer" containerID="db3af957e698ce780597fcf77d24d3f3875f6377ebe7e118fee16426a83b0020"
Sep 10 00:53:02.060234 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.060213    2563 scope.go:117] "RemoveContainer" containerID="800e0dea4b671f50b626e23ba0c553653435fbc3c4397ef014e1f0ce84b48142"
Sep 10 00:53:02.063771 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.063742    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 00:53:02.064147 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.064131    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="707530fa-8977-4428-b736-7f41881d20ec" containerName="thanos-sidecar"
Sep 10 00:53:02.064147 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.064147    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="707530fa-8977-4428-b736-7f41881d20ec" containerName="thanos-sidecar"
Sep 10 00:53:02.064278 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.064161    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cf5ecf8-4fd5-4c6e-99ea-4015ff995450" containerName="registry"
Sep 10 00:53:02.064278 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.064166    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cf5ecf8-4fd5-4c6e-99ea-4015ff995450" containerName="registry"
Sep 10 00:53:02.064278 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.064177    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9f9e798-953a-490b-8582-099077dd16fd" containerName="console"
Sep 10 00:53:02.064278 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.064183    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9f9e798-953a-490b-8582-099077dd16fd" containerName="console"
Sep 10 00:53:02.064278 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.064195    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="707530fa-8977-4428-b736-7f41881d20ec" containerName="kube-rbac-proxy"
Sep 10 00:53:02.064278 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.064200    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="707530fa-8977-4428-b736-7f41881d20ec" containerName="kube-rbac-proxy"
Sep 10 00:53:02.064278 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.064206    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="707530fa-8977-4428-b736-7f41881d20ec" containerName="kube-rbac-proxy-thanos"
Sep 10 00:53:02.064278 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.064212    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="707530fa-8977-4428-b736-7f41881d20ec" containerName="kube-rbac-proxy-thanos"
Sep 10 00:53:02.064278 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.064219    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="707530fa-8977-4428-b736-7f41881d20ec" containerName="prometheus"
Sep 10 00:53:02.064278 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.064224    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="707530fa-8977-4428-b736-7f41881d20ec" containerName="prometheus"
Sep 10 00:53:02.064278 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.064229    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="707530fa-8977-4428-b736-7f41881d20ec" containerName="kube-rbac-proxy-web"
Sep 10 00:53:02.064278 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.064234    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="707530fa-8977-4428-b736-7f41881d20ec" containerName="kube-rbac-proxy-web"
Sep 10 00:53:02.064278 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.064242    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="707530fa-8977-4428-b736-7f41881d20ec" containerName="init-config-reloader"
Sep 10 00:53:02.064278 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.064247    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="707530fa-8977-4428-b736-7f41881d20ec" containerName="init-config-reloader"
Sep 10 00:53:02.064278 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.064254    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="707530fa-8977-4428-b736-7f41881d20ec" containerName="config-reloader"
Sep 10 00:53:02.064278 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.064259    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="707530fa-8977-4428-b736-7f41881d20ec" containerName="config-reloader"
Sep 10 00:53:02.064904 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.064322    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="707530fa-8977-4428-b736-7f41881d20ec" containerName="prometheus"
Sep 10 00:53:02.064904 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.064334    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="707530fa-8977-4428-b736-7f41881d20ec" containerName="kube-rbac-proxy"
Sep 10 00:53:02.064904 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.064344    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9f9e798-953a-490b-8582-099077dd16fd" containerName="console"
Sep 10 00:53:02.064904 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.064354    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="707530fa-8977-4428-b736-7f41881d20ec" containerName="kube-rbac-proxy-web"
Sep 10 00:53:02.064904 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.064362    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="707530fa-8977-4428-b736-7f41881d20ec" containerName="config-reloader"
Sep 10 00:53:02.064904 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.064367    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="707530fa-8977-4428-b736-7f41881d20ec" containerName="thanos-sidecar"
Sep 10 00:53:02.064904 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.064373    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cf5ecf8-4fd5-4c6e-99ea-4015ff995450" containerName="registry"
Sep 10 00:53:02.064904 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.064377    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="707530fa-8977-4428-b736-7f41881d20ec" containerName="kube-rbac-proxy-thanos"
Sep 10 00:53:02.068232 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.068201    2563 scope.go:117] "RemoveContainer" containerID="50dde9009ec5ae128e15464153db8d75083cedede59401844e03ccdb626b4443"
Sep 10 00:53:02.068886 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:53:02.068861    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50dde9009ec5ae128e15464153db8d75083cedede59401844e03ccdb626b4443\": container with ID starting with 50dde9009ec5ae128e15464153db8d75083cedede59401844e03ccdb626b4443 not found: ID does not exist" containerID="50dde9009ec5ae128e15464153db8d75083cedede59401844e03ccdb626b4443"
Sep 10 00:53:02.068964 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.068892    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50dde9009ec5ae128e15464153db8d75083cedede59401844e03ccdb626b4443"} err="failed to get container status \"50dde9009ec5ae128e15464153db8d75083cedede59401844e03ccdb626b4443\": rpc error: code = NotFound desc = could not find container \"50dde9009ec5ae128e15464153db8d75083cedede59401844e03ccdb626b4443\": container with ID starting with 50dde9009ec5ae128e15464153db8d75083cedede59401844e03ccdb626b4443 not found: ID does not exist"
Sep 10 00:53:02.068964 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.068913    2563 scope.go:117] "RemoveContainer" containerID="8a26d97e8a3b0c032408178b84047e58dec1186b2b5df21410d58cc0ab6ce19b"
Sep 10 00:53:02.069322 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:53:02.069304    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a26d97e8a3b0c032408178b84047e58dec1186b2b5df21410d58cc0ab6ce19b\": container with ID starting with 8a26d97e8a3b0c032408178b84047e58dec1186b2b5df21410d58cc0ab6ce19b not found: ID does not exist" containerID="8a26d97e8a3b0c032408178b84047e58dec1186b2b5df21410d58cc0ab6ce19b"
Sep 10 00:53:02.069468 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.069328    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a26d97e8a3b0c032408178b84047e58dec1186b2b5df21410d58cc0ab6ce19b"} err="failed to get container status \"8a26d97e8a3b0c032408178b84047e58dec1186b2b5df21410d58cc0ab6ce19b\": rpc error: code = NotFound desc = could not find container \"8a26d97e8a3b0c032408178b84047e58dec1186b2b5df21410d58cc0ab6ce19b\": container with ID starting with 8a26d97e8a3b0c032408178b84047e58dec1186b2b5df21410d58cc0ab6ce19b not found: ID does not exist"
Sep 10 00:53:02.069468 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.069343    2563 scope.go:117] "RemoveContainer" containerID="7d908bda0bb1ea306b0f1f365fce752bf2a546a61301c387c37a54dae9d0bcd6"
Sep 10 00:53:02.069559 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:53:02.069543    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d908bda0bb1ea306b0f1f365fce752bf2a546a61301c387c37a54dae9d0bcd6\": container with ID starting with 7d908bda0bb1ea306b0f1f365fce752bf2a546a61301c387c37a54dae9d0bcd6 not found: ID does not exist" containerID="7d908bda0bb1ea306b0f1f365fce752bf2a546a61301c387c37a54dae9d0bcd6"
Sep 10 00:53:02.069604 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.069559    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d908bda0bb1ea306b0f1f365fce752bf2a546a61301c387c37a54dae9d0bcd6"} err="failed to get container status \"7d908bda0bb1ea306b0f1f365fce752bf2a546a61301c387c37a54dae9d0bcd6\": rpc error: code = NotFound desc = could not find container \"7d908bda0bb1ea306b0f1f365fce752bf2a546a61301c387c37a54dae9d0bcd6\": container with ID starting with 7d908bda0bb1ea306b0f1f365fce752bf2a546a61301c387c37a54dae9d0bcd6 not found: ID does not exist"
Sep 10 00:53:02.069604 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.069572    2563 scope.go:117] "RemoveContainer" containerID="2d283fc4d6ab716a6dedb56d0be2982fc8c91056e9af387facf4e85d81417334"
Sep 10 00:53:02.069789 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:53:02.069770    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d283fc4d6ab716a6dedb56d0be2982fc8c91056e9af387facf4e85d81417334\": container with ID starting with 2d283fc4d6ab716a6dedb56d0be2982fc8c91056e9af387facf4e85d81417334 not found: ID does not exist" containerID="2d283fc4d6ab716a6dedb56d0be2982fc8c91056e9af387facf4e85d81417334"
Sep 10 00:53:02.069831 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.069794    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d283fc4d6ab716a6dedb56d0be2982fc8c91056e9af387facf4e85d81417334"} err="failed to get container status \"2d283fc4d6ab716a6dedb56d0be2982fc8c91056e9af387facf4e85d81417334\": rpc error: code = NotFound desc = could not find container \"2d283fc4d6ab716a6dedb56d0be2982fc8c91056e9af387facf4e85d81417334\": container with ID starting with 2d283fc4d6ab716a6dedb56d0be2982fc8c91056e9af387facf4e85d81417334 not found: ID does not exist"
Sep 10 00:53:02.069831 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.069809    2563 scope.go:117] "RemoveContainer" containerID="e40d01213ad66004c11e521003c94dcc6e99b2399c6f0607cda58a51a2089555"
Sep 10 00:53:02.070045 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.070032    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.070118 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:53:02.070034    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e40d01213ad66004c11e521003c94dcc6e99b2399c6f0607cda58a51a2089555\": container with ID starting with e40d01213ad66004c11e521003c94dcc6e99b2399c6f0607cda58a51a2089555 not found: ID does not exist" containerID="e40d01213ad66004c11e521003c94dcc6e99b2399c6f0607cda58a51a2089555"
Sep 10 00:53:02.070182 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.070119    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e40d01213ad66004c11e521003c94dcc6e99b2399c6f0607cda58a51a2089555"} err="failed to get container status \"e40d01213ad66004c11e521003c94dcc6e99b2399c6f0607cda58a51a2089555\": rpc error: code = NotFound desc = could not find container \"e40d01213ad66004c11e521003c94dcc6e99b2399c6f0607cda58a51a2089555\": container with ID starting with e40d01213ad66004c11e521003c94dcc6e99b2399c6f0607cda58a51a2089555 not found: ID does not exist"
Sep 10 00:53:02.070182 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.070132    2563 scope.go:117] "RemoveContainer" containerID="db3af957e698ce780597fcf77d24d3f3875f6377ebe7e118fee16426a83b0020"
Sep 10 00:53:02.070550 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:53:02.070530    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db3af957e698ce780597fcf77d24d3f3875f6377ebe7e118fee16426a83b0020\": container with ID starting with db3af957e698ce780597fcf77d24d3f3875f6377ebe7e118fee16426a83b0020 not found: ID does not exist" containerID="db3af957e698ce780597fcf77d24d3f3875f6377ebe7e118fee16426a83b0020"
Sep 10 00:53:02.070664 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.070554    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db3af957e698ce780597fcf77d24d3f3875f6377ebe7e118fee16426a83b0020"} err="failed to get container status \"db3af957e698ce780597fcf77d24d3f3875f6377ebe7e118fee16426a83b0020\": rpc error: code = NotFound desc = could not find container \"db3af957e698ce780597fcf77d24d3f3875f6377ebe7e118fee16426a83b0020\": container with ID starting with db3af957e698ce780597fcf77d24d3f3875f6377ebe7e118fee16426a83b0020 not found: ID does not exist"
Sep 10 00:53:02.070664 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.070576    2563 scope.go:117] "RemoveContainer" containerID="800e0dea4b671f50b626e23ba0c553653435fbc3c4397ef014e1f0ce84b48142"
Sep 10 00:53:02.070835 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:53:02.070819    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"800e0dea4b671f50b626e23ba0c553653435fbc3c4397ef014e1f0ce84b48142\": container with ID starting with 800e0dea4b671f50b626e23ba0c553653435fbc3c4397ef014e1f0ce84b48142 not found: ID does not exist" containerID="800e0dea4b671f50b626e23ba0c553653435fbc3c4397ef014e1f0ce84b48142"
Sep 10 00:53:02.070893 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.070837    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"800e0dea4b671f50b626e23ba0c553653435fbc3c4397ef014e1f0ce84b48142"} err="failed to get container status \"800e0dea4b671f50b626e23ba0c553653435fbc3c4397ef014e1f0ce84b48142\": rpc error: code = NotFound desc = could not find container \"800e0dea4b671f50b626e23ba0c553653435fbc3c4397ef014e1f0ce84b48142\": container with ID starting with 800e0dea4b671f50b626e23ba0c553653435fbc3c4397ef014e1f0ce84b48142 not found: ID does not exist"
Sep 10 00:53:02.070893 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.070849    2563 scope.go:117] "RemoveContainer" containerID="50dde9009ec5ae128e15464153db8d75083cedede59401844e03ccdb626b4443"
Sep 10 00:53:02.071112 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.071093    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50dde9009ec5ae128e15464153db8d75083cedede59401844e03ccdb626b4443"} err="failed to get container status \"50dde9009ec5ae128e15464153db8d75083cedede59401844e03ccdb626b4443\": rpc error: code = NotFound desc = could not find container \"50dde9009ec5ae128e15464153db8d75083cedede59401844e03ccdb626b4443\": container with ID starting with 50dde9009ec5ae128e15464153db8d75083cedede59401844e03ccdb626b4443 not found: ID does not exist"
Sep 10 00:53:02.071157 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.071116    2563 scope.go:117] "RemoveContainer" containerID="8a26d97e8a3b0c032408178b84047e58dec1186b2b5df21410d58cc0ab6ce19b"
Sep 10 00:53:02.071380 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.071363    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a26d97e8a3b0c032408178b84047e58dec1186b2b5df21410d58cc0ab6ce19b"} err="failed to get container status \"8a26d97e8a3b0c032408178b84047e58dec1186b2b5df21410d58cc0ab6ce19b\": rpc error: code = NotFound desc = could not find container \"8a26d97e8a3b0c032408178b84047e58dec1186b2b5df21410d58cc0ab6ce19b\": container with ID starting with 8a26d97e8a3b0c032408178b84047e58dec1186b2b5df21410d58cc0ab6ce19b not found: ID does not exist"
Sep 10 00:53:02.071439 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.071381    2563 scope.go:117] "RemoveContainer" containerID="7d908bda0bb1ea306b0f1f365fce752bf2a546a61301c387c37a54dae9d0bcd6"
Sep 10 00:53:02.071659 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.071639    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d908bda0bb1ea306b0f1f365fce752bf2a546a61301c387c37a54dae9d0bcd6"} err="failed to get container status \"7d908bda0bb1ea306b0f1f365fce752bf2a546a61301c387c37a54dae9d0bcd6\": rpc error: code = NotFound desc = could not find container \"7d908bda0bb1ea306b0f1f365fce752bf2a546a61301c387c37a54dae9d0bcd6\": container with ID starting with 7d908bda0bb1ea306b0f1f365fce752bf2a546a61301c387c37a54dae9d0bcd6 not found: ID does not exist"
Sep 10 00:53:02.071659 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.071658    2563 scope.go:117] "RemoveContainer" containerID="2d283fc4d6ab716a6dedb56d0be2982fc8c91056e9af387facf4e85d81417334"
Sep 10 00:53:02.071954 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.071933    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d283fc4d6ab716a6dedb56d0be2982fc8c91056e9af387facf4e85d81417334"} err="failed to get container status \"2d283fc4d6ab716a6dedb56d0be2982fc8c91056e9af387facf4e85d81417334\": rpc error: code = NotFound desc = could not find container \"2d283fc4d6ab716a6dedb56d0be2982fc8c91056e9af387facf4e85d81417334\": container with ID starting with 2d283fc4d6ab716a6dedb56d0be2982fc8c91056e9af387facf4e85d81417334 not found: ID does not exist"
Sep 10 00:53:02.071954 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.071952    2563 scope.go:117] "RemoveContainer" containerID="e40d01213ad66004c11e521003c94dcc6e99b2399c6f0607cda58a51a2089555"
Sep 10 00:53:02.072212 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.072189    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e40d01213ad66004c11e521003c94dcc6e99b2399c6f0607cda58a51a2089555"} err="failed to get container status \"e40d01213ad66004c11e521003c94dcc6e99b2399c6f0607cda58a51a2089555\": rpc error: code = NotFound desc = could not find container \"e40d01213ad66004c11e521003c94dcc6e99b2399c6f0607cda58a51a2089555\": container with ID starting with e40d01213ad66004c11e521003c94dcc6e99b2399c6f0607cda58a51a2089555 not found: ID does not exist"
Sep 10 00:53:02.072212 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.072211    2563 scope.go:117] "RemoveContainer" containerID="db3af957e698ce780597fcf77d24d3f3875f6377ebe7e118fee16426a83b0020"
Sep 10 00:53:02.072444 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.072421    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db3af957e698ce780597fcf77d24d3f3875f6377ebe7e118fee16426a83b0020"} err="failed to get container status \"db3af957e698ce780597fcf77d24d3f3875f6377ebe7e118fee16426a83b0020\": rpc error: code = NotFound desc = could not find container \"db3af957e698ce780597fcf77d24d3f3875f6377ebe7e118fee16426a83b0020\": container with ID starting with db3af957e698ce780597fcf77d24d3f3875f6377ebe7e118fee16426a83b0020 not found: ID does not exist"
Sep 10 00:53:02.072444 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.072442    2563 scope.go:117] "RemoveContainer" containerID="800e0dea4b671f50b626e23ba0c553653435fbc3c4397ef014e1f0ce84b48142"
Sep 10 00:53:02.072764 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.072744    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"800e0dea4b671f50b626e23ba0c553653435fbc3c4397ef014e1f0ce84b48142"} err="failed to get container status \"800e0dea4b671f50b626e23ba0c553653435fbc3c4397ef014e1f0ce84b48142\": rpc error: code = NotFound desc = could not find container \"800e0dea4b671f50b626e23ba0c553653435fbc3c4397ef014e1f0ce84b48142\": container with ID starting with 800e0dea4b671f50b626e23ba0c553653435fbc3c4397ef014e1f0ce84b48142 not found: ID does not exist"
Sep 10 00:53:02.072764 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.072762    2563 scope.go:117] "RemoveContainer" containerID="50dde9009ec5ae128e15464153db8d75083cedede59401844e03ccdb626b4443"
Sep 10 00:53:02.073113 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.073050    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50dde9009ec5ae128e15464153db8d75083cedede59401844e03ccdb626b4443"} err="failed to get container status \"50dde9009ec5ae128e15464153db8d75083cedede59401844e03ccdb626b4443\": rpc error: code = NotFound desc = could not find container \"50dde9009ec5ae128e15464153db8d75083cedede59401844e03ccdb626b4443\": container with ID starting with 50dde9009ec5ae128e15464153db8d75083cedede59401844e03ccdb626b4443 not found: ID does not exist"
Sep 10 00:53:02.073113 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.073084    2563 scope.go:117] "RemoveContainer" containerID="8a26d97e8a3b0c032408178b84047e58dec1186b2b5df21410d58cc0ab6ce19b"
Sep 10 00:53:02.073251 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.073126    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 10 00:53:02.073251 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.073130    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 10 00:53:02.073486 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.073461    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a26d97e8a3b0c032408178b84047e58dec1186b2b5df21410d58cc0ab6ce19b"} err="failed to get container status \"8a26d97e8a3b0c032408178b84047e58dec1186b2b5df21410d58cc0ab6ce19b\": rpc error: code = NotFound desc = could not find container \"8a26d97e8a3b0c032408178b84047e58dec1186b2b5df21410d58cc0ab6ce19b\": container with ID starting with 8a26d97e8a3b0c032408178b84047e58dec1186b2b5df21410d58cc0ab6ce19b not found: ID does not exist"
Sep 10 00:53:02.073554 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.073490    2563 scope.go:117] "RemoveContainer" containerID="7d908bda0bb1ea306b0f1f365fce752bf2a546a61301c387c37a54dae9d0bcd6"
Sep 10 00:53:02.073624 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.073608    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 10 00:53:02.073683 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.073609    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 10 00:53:02.073729 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.073713    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 10 00:53:02.073781 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.073761    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d908bda0bb1ea306b0f1f365fce752bf2a546a61301c387c37a54dae9d0bcd6"} err="failed to get container status \"7d908bda0bb1ea306b0f1f365fce752bf2a546a61301c387c37a54dae9d0bcd6\": rpc error: code = NotFound desc = could not find container \"7d908bda0bb1ea306b0f1f365fce752bf2a546a61301c387c37a54dae9d0bcd6\": container with ID starting with 7d908bda0bb1ea306b0f1f365fce752bf2a546a61301c387c37a54dae9d0bcd6 not found: ID does not exist"
Sep 10 00:53:02.073827 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.073786    2563 scope.go:117] "RemoveContainer" containerID="2d283fc4d6ab716a6dedb56d0be2982fc8c91056e9af387facf4e85d81417334"
Sep 10 00:53:02.074051 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.074008    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 10 00:53:02.074132 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.074110    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d283fc4d6ab716a6dedb56d0be2982fc8c91056e9af387facf4e85d81417334"} err="failed to get container status \"2d283fc4d6ab716a6dedb56d0be2982fc8c91056e9af387facf4e85d81417334\": rpc error: code = NotFound desc = could not find container \"2d283fc4d6ab716a6dedb56d0be2982fc8c91056e9af387facf4e85d81417334\": container with ID starting with 2d283fc4d6ab716a6dedb56d0be2982fc8c91056e9af387facf4e85d81417334 not found: ID does not exist"
Sep 10 00:53:02.074195 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.074134    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 10 00:53:02.074195 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.074137    2563 scope.go:117] "RemoveContainer" containerID="e40d01213ad66004c11e521003c94dcc6e99b2399c6f0607cda58a51a2089555"
Sep 10 00:53:02.074460 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.074430    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e40d01213ad66004c11e521003c94dcc6e99b2399c6f0607cda58a51a2089555"} err="failed to get container status \"e40d01213ad66004c11e521003c94dcc6e99b2399c6f0607cda58a51a2089555\": rpc error: code = NotFound desc = could not find container \"e40d01213ad66004c11e521003c94dcc6e99b2399c6f0607cda58a51a2089555\": container with ID starting with e40d01213ad66004c11e521003c94dcc6e99b2399c6f0607cda58a51a2089555 not found: ID does not exist"
Sep 10 00:53:02.074460 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.074461    2563 scope.go:117] "RemoveContainer" containerID="db3af957e698ce780597fcf77d24d3f3875f6377ebe7e118fee16426a83b0020"
Sep 10 00:53:02.074753 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.074726    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db3af957e698ce780597fcf77d24d3f3875f6377ebe7e118fee16426a83b0020"} err="failed to get container status \"db3af957e698ce780597fcf77d24d3f3875f6377ebe7e118fee16426a83b0020\": rpc error: code = NotFound desc = could not find container \"db3af957e698ce780597fcf77d24d3f3875f6377ebe7e118fee16426a83b0020\": container with ID starting with db3af957e698ce780597fcf77d24d3f3875f6377ebe7e118fee16426a83b0020 not found: ID does not exist"
Sep 10 00:53:02.074834 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.074754    2563 scope.go:117] "RemoveContainer" containerID="800e0dea4b671f50b626e23ba0c553653435fbc3c4397ef014e1f0ce84b48142"
Sep 10 00:53:02.075063 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.075044    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"800e0dea4b671f50b626e23ba0c553653435fbc3c4397ef014e1f0ce84b48142"} err="failed to get container status \"800e0dea4b671f50b626e23ba0c553653435fbc3c4397ef014e1f0ce84b48142\": rpc error: code = NotFound desc = could not find container \"800e0dea4b671f50b626e23ba0c553653435fbc3c4397ef014e1f0ce84b48142\": container with ID starting with 800e0dea4b671f50b626e23ba0c553653435fbc3c4397ef014e1f0ce84b48142 not found: ID does not exist"
Sep 10 00:53:02.075135 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.075065    2563 scope.go:117] "RemoveContainer" containerID="50dde9009ec5ae128e15464153db8d75083cedede59401844e03ccdb626b4443"
Sep 10 00:53:02.075343 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.075320    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50dde9009ec5ae128e15464153db8d75083cedede59401844e03ccdb626b4443"} err="failed to get container status \"50dde9009ec5ae128e15464153db8d75083cedede59401844e03ccdb626b4443\": rpc error: code = NotFound desc = could not find container \"50dde9009ec5ae128e15464153db8d75083cedede59401844e03ccdb626b4443\": container with ID starting with 50dde9009ec5ae128e15464153db8d75083cedede59401844e03ccdb626b4443 not found: ID does not exist"
Sep 10 00:53:02.075426 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.075346    2563 scope.go:117] "RemoveContainer" containerID="8a26d97e8a3b0c032408178b84047e58dec1186b2b5df21410d58cc0ab6ce19b"
Sep 10 00:53:02.075606 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.075590    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a26d97e8a3b0c032408178b84047e58dec1186b2b5df21410d58cc0ab6ce19b"} err="failed to get container status \"8a26d97e8a3b0c032408178b84047e58dec1186b2b5df21410d58cc0ab6ce19b\": rpc error: code = NotFound desc = could not find container \"8a26d97e8a3b0c032408178b84047e58dec1186b2b5df21410d58cc0ab6ce19b\": container with ID starting with 8a26d97e8a3b0c032408178b84047e58dec1186b2b5df21410d58cc0ab6ce19b not found: ID does not exist"
Sep 10 00:53:02.075671 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.075606    2563 scope.go:117] "RemoveContainer" containerID="7d908bda0bb1ea306b0f1f365fce752bf2a546a61301c387c37a54dae9d0bcd6"
Sep 10 00:53:02.075877 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.075855    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d908bda0bb1ea306b0f1f365fce752bf2a546a61301c387c37a54dae9d0bcd6"} err="failed to get container status \"7d908bda0bb1ea306b0f1f365fce752bf2a546a61301c387c37a54dae9d0bcd6\": rpc error: code = NotFound desc = could not find container \"7d908bda0bb1ea306b0f1f365fce752bf2a546a61301c387c37a54dae9d0bcd6\": container with ID starting with 7d908bda0bb1ea306b0f1f365fce752bf2a546a61301c387c37a54dae9d0bcd6 not found: ID does not exist"
Sep 10 00:53:02.075877 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.075876    2563 scope.go:117] "RemoveContainer" containerID="2d283fc4d6ab716a6dedb56d0be2982fc8c91056e9af387facf4e85d81417334"
Sep 10 00:53:02.076202 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.076161    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 10 00:53:02.076202 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.076181    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d283fc4d6ab716a6dedb56d0be2982fc8c91056e9af387facf4e85d81417334"} err="failed to get container status \"2d283fc4d6ab716a6dedb56d0be2982fc8c91056e9af387facf4e85d81417334\": rpc error: code = NotFound desc = could not find container \"2d283fc4d6ab716a6dedb56d0be2982fc8c91056e9af387facf4e85d81417334\": container with ID starting with 2d283fc4d6ab716a6dedb56d0be2982fc8c91056e9af387facf4e85d81417334 not found: ID does not exist"
Sep 10 00:53:02.076349 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.076206    2563 scope.go:117] "RemoveContainer" containerID="e40d01213ad66004c11e521003c94dcc6e99b2399c6f0607cda58a51a2089555"
Sep 10 00:53:02.076489 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.076470    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e40d01213ad66004c11e521003c94dcc6e99b2399c6f0607cda58a51a2089555"} err="failed to get container status \"e40d01213ad66004c11e521003c94dcc6e99b2399c6f0607cda58a51a2089555\": rpc error: code = NotFound desc = could not find container \"e40d01213ad66004c11e521003c94dcc6e99b2399c6f0607cda58a51a2089555\": container with ID starting with e40d01213ad66004c11e521003c94dcc6e99b2399c6f0607cda58a51a2089555 not found: ID does not exist"
Sep 10 00:53:02.076601 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.076490    2563 scope.go:117] "RemoveContainer" containerID="db3af957e698ce780597fcf77d24d3f3875f6377ebe7e118fee16426a83b0020"
Sep 10 00:53:02.076759 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.076738    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db3af957e698ce780597fcf77d24d3f3875f6377ebe7e118fee16426a83b0020"} err="failed to get container status \"db3af957e698ce780597fcf77d24d3f3875f6377ebe7e118fee16426a83b0020\": rpc error: code = NotFound desc = could not find container \"db3af957e698ce780597fcf77d24d3f3875f6377ebe7e118fee16426a83b0020\": container with ID starting with db3af957e698ce780597fcf77d24d3f3875f6377ebe7e118fee16426a83b0020 not found: ID does not exist"
Sep 10 00:53:02.076806 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.076760    2563 scope.go:117] "RemoveContainer" containerID="800e0dea4b671f50b626e23ba0c553653435fbc3c4397ef014e1f0ce84b48142"
Sep 10 00:53:02.077016 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.076997    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 10 00:53:02.077016 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.077011    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 10 00:53:02.077138 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.077030    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"800e0dea4b671f50b626e23ba0c553653435fbc3c4397ef014e1f0ce84b48142"} err="failed to get container status \"800e0dea4b671f50b626e23ba0c553653435fbc3c4397ef014e1f0ce84b48142\": rpc error: code = NotFound desc = could not find container \"800e0dea4b671f50b626e23ba0c553653435fbc3c4397ef014e1f0ce84b48142\": container with ID starting with 800e0dea4b671f50b626e23ba0c553653435fbc3c4397ef014e1f0ce84b48142 not found: ID does not exist"
Sep 10 00:53:02.077138 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.077052    2563 scope.go:117] "RemoveContainer" containerID="50dde9009ec5ae128e15464153db8d75083cedede59401844e03ccdb626b4443"
Sep 10 00:53:02.077264 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.077247    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 10 00:53:02.077362 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.077347    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 10 00:53:02.077481 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.077463    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2godtq8b27jq3\""
Sep 10 00:53:02.077575 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.077463    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 10 00:53:02.077575 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.077357    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50dde9009ec5ae128e15464153db8d75083cedede59401844e03ccdb626b4443"} err="failed to get container status \"50dde9009ec5ae128e15464153db8d75083cedede59401844e03ccdb626b4443\": rpc error: code = NotFound desc = could not find container \"50dde9009ec5ae128e15464153db8d75083cedede59401844e03ccdb626b4443\": container with ID starting with 50dde9009ec5ae128e15464153db8d75083cedede59401844e03ccdb626b4443 not found: ID does not exist"
Sep 10 00:53:02.077575 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.077523    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-gpgd7\""
Sep 10 00:53:02.077575 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.077526    2563 scope.go:117] "RemoveContainer" containerID="8a26d97e8a3b0c032408178b84047e58dec1186b2b5df21410d58cc0ab6ce19b"
Sep 10 00:53:02.077885 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.077849    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a26d97e8a3b0c032408178b84047e58dec1186b2b5df21410d58cc0ab6ce19b"} err="failed to get container status \"8a26d97e8a3b0c032408178b84047e58dec1186b2b5df21410d58cc0ab6ce19b\": rpc error: code = NotFound desc = could not find container \"8a26d97e8a3b0c032408178b84047e58dec1186b2b5df21410d58cc0ab6ce19b\": container with ID starting with 8a26d97e8a3b0c032408178b84047e58dec1186b2b5df21410d58cc0ab6ce19b not found: ID does not exist"
Sep 10 00:53:02.077885 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.077879    2563 scope.go:117] "RemoveContainer" containerID="7d908bda0bb1ea306b0f1f365fce752bf2a546a61301c387c37a54dae9d0bcd6"
Sep 10 00:53:02.078216 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.078194    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d908bda0bb1ea306b0f1f365fce752bf2a546a61301c387c37a54dae9d0bcd6"} err="failed to get container status \"7d908bda0bb1ea306b0f1f365fce752bf2a546a61301c387c37a54dae9d0bcd6\": rpc error: code = NotFound desc = could not find container \"7d908bda0bb1ea306b0f1f365fce752bf2a546a61301c387c37a54dae9d0bcd6\": container with ID starting with 7d908bda0bb1ea306b0f1f365fce752bf2a546a61301c387c37a54dae9d0bcd6 not found: ID does not exist"
Sep 10 00:53:02.078305 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.078217    2563 scope.go:117] "RemoveContainer" containerID="2d283fc4d6ab716a6dedb56d0be2982fc8c91056e9af387facf4e85d81417334"
Sep 10 00:53:02.078522 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.078500    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d283fc4d6ab716a6dedb56d0be2982fc8c91056e9af387facf4e85d81417334"} err="failed to get container status \"2d283fc4d6ab716a6dedb56d0be2982fc8c91056e9af387facf4e85d81417334\": rpc error: code = NotFound desc = could not find container \"2d283fc4d6ab716a6dedb56d0be2982fc8c91056e9af387facf4e85d81417334\": container with ID starting with 2d283fc4d6ab716a6dedb56d0be2982fc8c91056e9af387facf4e85d81417334 not found: ID does not exist"
Sep 10 00:53:02.078620 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.078523    2563 scope.go:117] "RemoveContainer" containerID="e40d01213ad66004c11e521003c94dcc6e99b2399c6f0607cda58a51a2089555"
Sep 10 00:53:02.079333 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.079305    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e40d01213ad66004c11e521003c94dcc6e99b2399c6f0607cda58a51a2089555"} err="failed to get container status \"e40d01213ad66004c11e521003c94dcc6e99b2399c6f0607cda58a51a2089555\": rpc error: code = NotFound desc = could not find container \"e40d01213ad66004c11e521003c94dcc6e99b2399c6f0607cda58a51a2089555\": container with ID starting with e40d01213ad66004c11e521003c94dcc6e99b2399c6f0607cda58a51a2089555 not found: ID does not exist"
Sep 10 00:53:02.079429 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.079337    2563 scope.go:117] "RemoveContainer" containerID="db3af957e698ce780597fcf77d24d3f3875f6377ebe7e118fee16426a83b0020"
Sep 10 00:53:02.079780 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.079750    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 10 00:53:02.080189 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.080162    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db3af957e698ce780597fcf77d24d3f3875f6377ebe7e118fee16426a83b0020"} err="failed to get container status \"db3af957e698ce780597fcf77d24d3f3875f6377ebe7e118fee16426a83b0020\": rpc error: code = NotFound desc = could not find container \"db3af957e698ce780597fcf77d24d3f3875f6377ebe7e118fee16426a83b0020\": container with ID starting with db3af957e698ce780597fcf77d24d3f3875f6377ebe7e118fee16426a83b0020 not found: ID does not exist"
Sep 10 00:53:02.080297 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.080191    2563 scope.go:117] "RemoveContainer" containerID="800e0dea4b671f50b626e23ba0c553653435fbc3c4397ef014e1f0ce84b48142"
Sep 10 00:53:02.080483 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.080447    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"800e0dea4b671f50b626e23ba0c553653435fbc3c4397ef014e1f0ce84b48142"} err="failed to get container status \"800e0dea4b671f50b626e23ba0c553653435fbc3c4397ef014e1f0ce84b48142\": rpc error: code = NotFound desc = could not find container \"800e0dea4b671f50b626e23ba0c553653435fbc3c4397ef014e1f0ce84b48142\": container with ID starting with 800e0dea4b671f50b626e23ba0c553653435fbc3c4397ef014e1f0ce84b48142 not found: ID does not exist"
Sep 10 00:53:02.080545 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.080485    2563 scope.go:117] "RemoveContainer" containerID="50dde9009ec5ae128e15464153db8d75083cedede59401844e03ccdb626b4443"
Sep 10 00:53:02.080822 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.080796    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50dde9009ec5ae128e15464153db8d75083cedede59401844e03ccdb626b4443"} err="failed to get container status \"50dde9009ec5ae128e15464153db8d75083cedede59401844e03ccdb626b4443\": rpc error: code = NotFound desc = could not find container \"50dde9009ec5ae128e15464153db8d75083cedede59401844e03ccdb626b4443\": container with ID starting with 50dde9009ec5ae128e15464153db8d75083cedede59401844e03ccdb626b4443 not found: ID does not exist"
Sep 10 00:53:02.080822 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.080824    2563 scope.go:117] "RemoveContainer" containerID="8a26d97e8a3b0c032408178b84047e58dec1186b2b5df21410d58cc0ab6ce19b"
Sep 10 00:53:02.081194 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.081177    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 10 00:53:02.081274 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.081173    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a26d97e8a3b0c032408178b84047e58dec1186b2b5df21410d58cc0ab6ce19b"} err="failed to get container status \"8a26d97e8a3b0c032408178b84047e58dec1186b2b5df21410d58cc0ab6ce19b\": rpc error: code = NotFound desc = could not find container \"8a26d97e8a3b0c032408178b84047e58dec1186b2b5df21410d58cc0ab6ce19b\": container with ID starting with 8a26d97e8a3b0c032408178b84047e58dec1186b2b5df21410d58cc0ab6ce19b not found: ID does not exist"
Sep 10 00:53:02.081274 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.081218    2563 scope.go:117] "RemoveContainer" containerID="7d908bda0bb1ea306b0f1f365fce752bf2a546a61301c387c37a54dae9d0bcd6"
Sep 10 00:53:02.081506 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.081485    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d908bda0bb1ea306b0f1f365fce752bf2a546a61301c387c37a54dae9d0bcd6"} err="failed to get container status \"7d908bda0bb1ea306b0f1f365fce752bf2a546a61301c387c37a54dae9d0bcd6\": rpc error: code = NotFound desc = could not find container \"7d908bda0bb1ea306b0f1f365fce752bf2a546a61301c387c37a54dae9d0bcd6\": container with ID starting with 7d908bda0bb1ea306b0f1f365fce752bf2a546a61301c387c37a54dae9d0bcd6 not found: ID does not exist"
Sep 10 00:53:02.081595 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.081506    2563 scope.go:117] "RemoveContainer" containerID="2d283fc4d6ab716a6dedb56d0be2982fc8c91056e9af387facf4e85d81417334"
Sep 10 00:53:02.081762 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.081741    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d283fc4d6ab716a6dedb56d0be2982fc8c91056e9af387facf4e85d81417334"} err="failed to get container status \"2d283fc4d6ab716a6dedb56d0be2982fc8c91056e9af387facf4e85d81417334\": rpc error: code = NotFound desc = could not find container \"2d283fc4d6ab716a6dedb56d0be2982fc8c91056e9af387facf4e85d81417334\": container with ID starting with 2d283fc4d6ab716a6dedb56d0be2982fc8c91056e9af387facf4e85d81417334 not found: ID does not exist"
Sep 10 00:53:02.081762 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.081761    2563 scope.go:117] "RemoveContainer" containerID="e40d01213ad66004c11e521003c94dcc6e99b2399c6f0607cda58a51a2089555"
Sep 10 00:53:02.082035 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.082017    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e40d01213ad66004c11e521003c94dcc6e99b2399c6f0607cda58a51a2089555"} err="failed to get container status \"e40d01213ad66004c11e521003c94dcc6e99b2399c6f0607cda58a51a2089555\": rpc error: code = NotFound desc = could not find container \"e40d01213ad66004c11e521003c94dcc6e99b2399c6f0607cda58a51a2089555\": container with ID starting with e40d01213ad66004c11e521003c94dcc6e99b2399c6f0607cda58a51a2089555 not found: ID does not exist"
Sep 10 00:53:02.082102 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.082036    2563 scope.go:117] "RemoveContainer" containerID="db3af957e698ce780597fcf77d24d3f3875f6377ebe7e118fee16426a83b0020"
Sep 10 00:53:02.082261 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.082242    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db3af957e698ce780597fcf77d24d3f3875f6377ebe7e118fee16426a83b0020"} err="failed to get container status \"db3af957e698ce780597fcf77d24d3f3875f6377ebe7e118fee16426a83b0020\": rpc error: code = NotFound desc = could not find container \"db3af957e698ce780597fcf77d24d3f3875f6377ebe7e118fee16426a83b0020\": container with ID starting with db3af957e698ce780597fcf77d24d3f3875f6377ebe7e118fee16426a83b0020 not found: ID does not exist"
Sep 10 00:53:02.082261 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.082260    2563 scope.go:117] "RemoveContainer" containerID="800e0dea4b671f50b626e23ba0c553653435fbc3c4397ef014e1f0ce84b48142"
Sep 10 00:53:02.082500 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.082480    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"800e0dea4b671f50b626e23ba0c553653435fbc3c4397ef014e1f0ce84b48142"} err="failed to get container status \"800e0dea4b671f50b626e23ba0c553653435fbc3c4397ef014e1f0ce84b48142\": rpc error: code = NotFound desc = could not find container \"800e0dea4b671f50b626e23ba0c553653435fbc3c4397ef014e1f0ce84b48142\": container with ID starting with 800e0dea4b671f50b626e23ba0c553653435fbc3c4397ef014e1f0ce84b48142 not found: ID does not exist"
Sep 10 00:53:02.083458 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.083440    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="707530fa-8977-4428-b736-7f41881d20ec" path="/var/lib/kubelet/pods/707530fa-8977-4428-b736-7f41881d20ec/volumes"
Sep 10 00:53:02.086699 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.086676    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 00:53:02.092110 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.092087    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b629bec7-e856-4a92-afe8-df3dba62dbb9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.092206 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.092124    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b629bec7-e856-4a92-afe8-df3dba62dbb9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.092206 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.092149    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prctr\" (UniqueName: \"kubernetes.io/projected/b629bec7-e856-4a92-afe8-df3dba62dbb9-kube-api-access-prctr\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.092281 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.092216    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b629bec7-e856-4a92-afe8-df3dba62dbb9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.092321 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.092275    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b629bec7-e856-4a92-afe8-df3dba62dbb9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.092321 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.092304    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b629bec7-e856-4a92-afe8-df3dba62dbb9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.092400 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.092342    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b629bec7-e856-4a92-afe8-df3dba62dbb9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.092400 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.092375    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b629bec7-e856-4a92-afe8-df3dba62dbb9-web-config\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.092480 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.092399    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b629bec7-e856-4a92-afe8-df3dba62dbb9-config-out\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.092480 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.092430    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b629bec7-e856-4a92-afe8-df3dba62dbb9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.092480 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.092454    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b629bec7-e856-4a92-afe8-df3dba62dbb9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.092612 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.092505    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b629bec7-e856-4a92-afe8-df3dba62dbb9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.092612 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.092528    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b629bec7-e856-4a92-afe8-df3dba62dbb9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.092612 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.092551    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b629bec7-e856-4a92-afe8-df3dba62dbb9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.092612 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.092576    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b629bec7-e856-4a92-afe8-df3dba62dbb9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.092782 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.092617    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b629bec7-e856-4a92-afe8-df3dba62dbb9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.092782 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.092666    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b629bec7-e856-4a92-afe8-df3dba62dbb9-config\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.092782 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.092698    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b629bec7-e856-4a92-afe8-df3dba62dbb9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.092782 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.092719    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b629bec7-e856-4a92-afe8-df3dba62dbb9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.092782 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.092749    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b629bec7-e856-4a92-afe8-df3dba62dbb9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.193189 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.193147    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b629bec7-e856-4a92-afe8-df3dba62dbb9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.193189 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.193190    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b629bec7-e856-4a92-afe8-df3dba62dbb9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.193386 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.193296    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b629bec7-e856-4a92-afe8-df3dba62dbb9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.193386 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.193322    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b629bec7-e856-4a92-afe8-df3dba62dbb9-web-config\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.193386 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.193348    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b629bec7-e856-4a92-afe8-df3dba62dbb9-config-out\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.193386 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.193377    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b629bec7-e856-4a92-afe8-df3dba62dbb9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.193600 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.193403    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b629bec7-e856-4a92-afe8-df3dba62dbb9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.193600 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.193452    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b629bec7-e856-4a92-afe8-df3dba62dbb9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.193864 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.193834    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b629bec7-e856-4a92-afe8-df3dba62dbb9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.193950 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.193838    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b629bec7-e856-4a92-afe8-df3dba62dbb9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.193950 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.193901    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b629bec7-e856-4a92-afe8-df3dba62dbb9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.193950 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.193934    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b629bec7-e856-4a92-afe8-df3dba62dbb9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.194153 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.193962    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b629bec7-e856-4a92-afe8-df3dba62dbb9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.194153 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.194035    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b629bec7-e856-4a92-afe8-df3dba62dbb9-config\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.194153 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.194074    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b629bec7-e856-4a92-afe8-df3dba62dbb9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.194153 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.194093    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b629bec7-e856-4a92-afe8-df3dba62dbb9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.194153 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.194116    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b629bec7-e856-4a92-afe8-df3dba62dbb9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.194153 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.194149    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b629bec7-e856-4a92-afe8-df3dba62dbb9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.194459 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.194182    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b629bec7-e856-4a92-afe8-df3dba62dbb9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.194459 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.194204    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prctr\" (UniqueName: \"kubernetes.io/projected/b629bec7-e856-4a92-afe8-df3dba62dbb9-kube-api-access-prctr\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.194459 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.194307    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b629bec7-e856-4a92-afe8-df3dba62dbb9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.195565 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.195532    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b629bec7-e856-4a92-afe8-df3dba62dbb9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.195675 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.195619    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b629bec7-e856-4a92-afe8-df3dba62dbb9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.196245 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.196218    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b629bec7-e856-4a92-afe8-df3dba62dbb9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.196496 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.196474    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b629bec7-e856-4a92-afe8-df3dba62dbb9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.196719 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.196589    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b629bec7-e856-4a92-afe8-df3dba62dbb9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.196719 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.196593    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b629bec7-e856-4a92-afe8-df3dba62dbb9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.198886 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.197212    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b629bec7-e856-4a92-afe8-df3dba62dbb9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.198886 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.197357    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b629bec7-e856-4a92-afe8-df3dba62dbb9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.198886 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.197499    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b629bec7-e856-4a92-afe8-df3dba62dbb9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.198886 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.197608    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b629bec7-e856-4a92-afe8-df3dba62dbb9-web-config\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.198886 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.197706    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b629bec7-e856-4a92-afe8-df3dba62dbb9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.198886 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.197837    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b629bec7-e856-4a92-afe8-df3dba62dbb9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.198886 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.198503    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b629bec7-e856-4a92-afe8-df3dba62dbb9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.199319 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.199084    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b629bec7-e856-4a92-afe8-df3dba62dbb9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.199425 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.199406    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/b629bec7-e856-4a92-afe8-df3dba62dbb9-config\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.199651 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.199634    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b629bec7-e856-4a92-afe8-df3dba62dbb9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.200324 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.200304    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b629bec7-e856-4a92-afe8-df3dba62dbb9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.200816 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.200801    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b629bec7-e856-4a92-afe8-df3dba62dbb9-config-out\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.205139 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.205120    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prctr\" (UniqueName: \"kubernetes.io/projected/b629bec7-e856-4a92-afe8-df3dba62dbb9-kube-api-access-prctr\") pod \"prometheus-k8s-0\" (UID: \"b629bec7-e856-4a92-afe8-df3dba62dbb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.382129 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.382095    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:02.668959 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:02.668931    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 00:53:02.670977 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:53:02.670946    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb629bec7_e856_4a92_afe8_df3dba62dbb9.slice/crio-598aa5986190e8d44a7658f983e32109a7d30a99640335f68b17685f7b327169 WatchSource:0}: Error finding container 598aa5986190e8d44a7658f983e32109a7d30a99640335f68b17685f7b327169: Status 404 returned error can't find the container with id 598aa5986190e8d44a7658f983e32109a7d30a99640335f68b17685f7b327169
Sep 10 00:53:03.008871 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:03.008783    2563 generic.go:356] "Generic (PLEG): container finished" podID="b629bec7-e856-4a92-afe8-df3dba62dbb9" containerID="45674be521326bbfb7755662fec08409281a956c5aa8226912cbd7bc0ccf112d" exitCode=0
Sep 10 00:53:03.009052 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:03.008884    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b629bec7-e856-4a92-afe8-df3dba62dbb9","Type":"ContainerDied","Data":"45674be521326bbfb7755662fec08409281a956c5aa8226912cbd7bc0ccf112d"}
Sep 10 00:53:03.009052 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:03.008955    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b629bec7-e856-4a92-afe8-df3dba62dbb9","Type":"ContainerStarted","Data":"598aa5986190e8d44a7658f983e32109a7d30a99640335f68b17685f7b327169"}
Sep 10 00:53:04.016614 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:04.016575    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b629bec7-e856-4a92-afe8-df3dba62dbb9","Type":"ContainerStarted","Data":"a68656af4da2cd1fc4e221da43c64994aa64cdef1f7dd54c1acb5ff3de2cf4c3"}
Sep 10 00:53:04.016958 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:04.016668    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b629bec7-e856-4a92-afe8-df3dba62dbb9","Type":"ContainerStarted","Data":"99cb31adc2de6db6f2d12ee8bacba535dc03d5023d937c10909835cfa6b3f8b0"}
Sep 10 00:53:04.016958 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:04.016702    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b629bec7-e856-4a92-afe8-df3dba62dbb9","Type":"ContainerStarted","Data":"24fe68f6c10a716aaae872111ae22d1940502c0da400a753b2428344bf848d5b"}
Sep 10 00:53:04.016958 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:04.016729    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b629bec7-e856-4a92-afe8-df3dba62dbb9","Type":"ContainerStarted","Data":"2e5b745528191a4c9c4ead3a3dfc228e7a64675442736c8a3734d5c706e2ec86"}
Sep 10 00:53:04.016958 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:04.016752    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b629bec7-e856-4a92-afe8-df3dba62dbb9","Type":"ContainerStarted","Data":"19d3420f47524f2a7957bf1651f60fe2760715cea003d91a7e828b35382dd8a9"}
Sep 10 00:53:04.016958 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:04.016780    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b629bec7-e856-4a92-afe8-df3dba62dbb9","Type":"ContainerStarted","Data":"daa691a1bdc53e5fc7ccf0ec2e0331dedc31bd21f2929c89d031be2ea504a6d2"}
Sep 10 00:53:04.044340 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:04.044288    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.026578681 podStartE2EDuration="2.044272716s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.694035ms" isStatefulPod=false podCreationTimestamp="2026-09-10 00:53:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:53:04.042098167 +0000 UTC m=+200.539397745" watchObservedRunningTime="2026-09-10 00:53:04.044272716 +0000 UTC m=+200.541572348"
Sep 10 00:53:07.383095 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:07.383042    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:53:23.576005 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:23.572794    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-765785bbb6-5tjbc"]
Sep 10 00:53:48.596565 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:48.596521    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-765785bbb6-5tjbc" podUID="c2a99f59-1632-4ec2-a233-bb22ce55f9a6" containerName="console" containerID="cri-o://df8f7d29ed3a2cfa739db303395832abd4514c3aa9acb5bb20b4b3508c4d90fe" gracePeriod=15
Sep 10 00:53:48.832431 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:48.832406    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-765785bbb6-5tjbc"
Sep 10 00:53:48.990755 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:48.990661    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-console-serving-cert\") pod \"c2a99f59-1632-4ec2-a233-bb22ce55f9a6\" (UID: \"c2a99f59-1632-4ec2-a233-bb22ce55f9a6\") "
Sep 10 00:53:48.990755 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:48.990708    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-service-ca\") pod \"c2a99f59-1632-4ec2-a233-bb22ce55f9a6\" (UID: \"c2a99f59-1632-4ec2-a233-bb22ce55f9a6\") "
Sep 10 00:53:48.991011 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:48.990765    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-oauth-serving-cert\") pod \"c2a99f59-1632-4ec2-a233-bb22ce55f9a6\" (UID: \"c2a99f59-1632-4ec2-a233-bb22ce55f9a6\") "
Sep 10 00:53:48.991011 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:48.990814    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-console-oauth-config\") pod \"c2a99f59-1632-4ec2-a233-bb22ce55f9a6\" (UID: \"c2a99f59-1632-4ec2-a233-bb22ce55f9a6\") "
Sep 10 00:53:48.991011 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:48.990839    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-trusted-ca-bundle\") pod \"c2a99f59-1632-4ec2-a233-bb22ce55f9a6\" (UID: \"c2a99f59-1632-4ec2-a233-bb22ce55f9a6\") "
Sep 10 00:53:48.991011 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:48.990870    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-console-config\") pod \"c2a99f59-1632-4ec2-a233-bb22ce55f9a6\" (UID: \"c2a99f59-1632-4ec2-a233-bb22ce55f9a6\") "
Sep 10 00:53:48.991011 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:48.990916    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mptww\" (UniqueName: \"kubernetes.io/projected/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-kube-api-access-mptww\") pod \"c2a99f59-1632-4ec2-a233-bb22ce55f9a6\" (UID: \"c2a99f59-1632-4ec2-a233-bb22ce55f9a6\") "
Sep 10 00:53:48.991343 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:48.991318    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-service-ca" (OuterVolumeSpecName: "service-ca") pod "c2a99f59-1632-4ec2-a233-bb22ce55f9a6" (UID: "c2a99f59-1632-4ec2-a233-bb22ce55f9a6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 00:53:48.991439 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:48.991366    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c2a99f59-1632-4ec2-a233-bb22ce55f9a6" (UID: "c2a99f59-1632-4ec2-a233-bb22ce55f9a6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 00:53:48.991439 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:48.991419    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c2a99f59-1632-4ec2-a233-bb22ce55f9a6" (UID: "c2a99f59-1632-4ec2-a233-bb22ce55f9a6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 00:53:48.991526 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:48.991440    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-console-config" (OuterVolumeSpecName: "console-config") pod "c2a99f59-1632-4ec2-a233-bb22ce55f9a6" (UID: "c2a99f59-1632-4ec2-a233-bb22ce55f9a6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 00:53:48.993311 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:48.993278    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-kube-api-access-mptww" (OuterVolumeSpecName: "kube-api-access-mptww") pod "c2a99f59-1632-4ec2-a233-bb22ce55f9a6" (UID: "c2a99f59-1632-4ec2-a233-bb22ce55f9a6"). InnerVolumeSpecName "kube-api-access-mptww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 00:53:48.993415 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:48.993393    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c2a99f59-1632-4ec2-a233-bb22ce55f9a6" (UID: "c2a99f59-1632-4ec2-a233-bb22ce55f9a6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 00:53:48.993460 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:48.993401    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c2a99f59-1632-4ec2-a233-bb22ce55f9a6" (UID: "c2a99f59-1632-4ec2-a233-bb22ce55f9a6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 00:53:49.092172 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:49.092128    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mptww\" (UniqueName: \"kubernetes.io/projected/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-kube-api-access-mptww\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:53:49.092172 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:49.092162    2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-console-serving-cert\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:53:49.092172 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:49.092172    2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-service-ca\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:53:49.092172 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:49.092181    2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-oauth-serving-cert\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:53:49.092491 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:49.092192    2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-console-oauth-config\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:53:49.092491 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:49.092200    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-trusted-ca-bundle\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:53:49.092491 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:49.092208    2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c2a99f59-1632-4ec2-a233-bb22ce55f9a6-console-config\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:53:49.193735 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:49.193701    2563 generic.go:356] "Generic (PLEG): container finished" podID="c2a99f59-1632-4ec2-a233-bb22ce55f9a6" containerID="df8f7d29ed3a2cfa739db303395832abd4514c3aa9acb5bb20b4b3508c4d90fe" exitCode=0
Sep 10 00:53:49.193901 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:49.193747    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-765785bbb6-5tjbc"
Sep 10 00:53:49.193901 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:49.193786    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-765785bbb6-5tjbc" event={"ID":"c2a99f59-1632-4ec2-a233-bb22ce55f9a6","Type":"ContainerDied","Data":"df8f7d29ed3a2cfa739db303395832abd4514c3aa9acb5bb20b4b3508c4d90fe"}
Sep 10 00:53:49.193901 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:49.193841    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-765785bbb6-5tjbc" event={"ID":"c2a99f59-1632-4ec2-a233-bb22ce55f9a6","Type":"ContainerDied","Data":"e11b5f5f850b02547b904cc45169197c417d5b49f74fec4133e822779c407bb7"}
Sep 10 00:53:49.193901 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:49.193870    2563 scope.go:117] "RemoveContainer" containerID="df8f7d29ed3a2cfa739db303395832abd4514c3aa9acb5bb20b4b3508c4d90fe"
Sep 10 00:53:49.202592 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:49.202569    2563 scope.go:117] "RemoveContainer" containerID="df8f7d29ed3a2cfa739db303395832abd4514c3aa9acb5bb20b4b3508c4d90fe"
Sep 10 00:53:49.202893 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:53:49.202872    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df8f7d29ed3a2cfa739db303395832abd4514c3aa9acb5bb20b4b3508c4d90fe\": container with ID starting with df8f7d29ed3a2cfa739db303395832abd4514c3aa9acb5bb20b4b3508c4d90fe not found: ID does not exist" containerID="df8f7d29ed3a2cfa739db303395832abd4514c3aa9acb5bb20b4b3508c4d90fe"
Sep 10 00:53:49.202958 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:49.202906    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df8f7d29ed3a2cfa739db303395832abd4514c3aa9acb5bb20b4b3508c4d90fe"} err="failed to get container status \"df8f7d29ed3a2cfa739db303395832abd4514c3aa9acb5bb20b4b3508c4d90fe\": rpc error: code = NotFound desc = could not find container \"df8f7d29ed3a2cfa739db303395832abd4514c3aa9acb5bb20b4b3508c4d90fe\": container with ID starting with df8f7d29ed3a2cfa739db303395832abd4514c3aa9acb5bb20b4b3508c4d90fe not found: ID does not exist"
Sep 10 00:53:49.214076 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:49.214052    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-765785bbb6-5tjbc"]
Sep 10 00:53:49.218058 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:49.218033    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-765785bbb6-5tjbc"]
Sep 10 00:53:50.081698 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:53:50.081663    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2a99f59-1632-4ec2-a233-bb22ce55f9a6" path="/var/lib/kubelet/pods/c2a99f59-1632-4ec2-a233-bb22ce55f9a6/volumes"
Sep 10 00:54:02.382694 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:02.382659    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:54:02.398212 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:02.398182    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:54:03.262202 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:03.262171    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 00:54:23.585847 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:23.585805    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-xm956"]
Sep 10 00:54:23.586407 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:23.586250    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2a99f59-1632-4ec2-a233-bb22ce55f9a6" containerName="console"
Sep 10 00:54:23.586407 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:23.586269    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2a99f59-1632-4ec2-a233-bb22ce55f9a6" containerName="console"
Sep 10 00:54:23.586407 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:23.586379    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2a99f59-1632-4ec2-a233-bb22ce55f9a6" containerName="console"
Sep 10 00:54:23.589820 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:23.589798    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-xm956"
Sep 10 00:54:23.592749 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:23.592726    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-8wscl\""
Sep 10 00:54:23.593413 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:23.593396    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 10 00:54:23.593644 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:23.593628    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 10 00:54:23.605397 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:23.605372    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-xm956"]
Sep 10 00:54:23.681946 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:23.681912    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62x42\" (UniqueName: \"kubernetes.io/projected/77e24e29-3230-4a3c-b824-bc57866b2831-kube-api-access-62x42\") pod \"cert-manager-webhook-6486ffd997-xm956\" (UID: \"77e24e29-3230-4a3c-b824-bc57866b2831\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xm956"
Sep 10 00:54:23.682148 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:23.681964    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/77e24e29-3230-4a3c-b824-bc57866b2831-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-xm956\" (UID: \"77e24e29-3230-4a3c-b824-bc57866b2831\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xm956"
Sep 10 00:54:23.783046 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:23.783004    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62x42\" (UniqueName: \"kubernetes.io/projected/77e24e29-3230-4a3c-b824-bc57866b2831-kube-api-access-62x42\") pod \"cert-manager-webhook-6486ffd997-xm956\" (UID: \"77e24e29-3230-4a3c-b824-bc57866b2831\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xm956"
Sep 10 00:54:23.783207 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:23.783077    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/77e24e29-3230-4a3c-b824-bc57866b2831-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-xm956\" (UID: \"77e24e29-3230-4a3c-b824-bc57866b2831\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xm956"
Sep 10 00:54:23.792359 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:23.792322    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/77e24e29-3230-4a3c-b824-bc57866b2831-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-xm956\" (UID: \"77e24e29-3230-4a3c-b824-bc57866b2831\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xm956"
Sep 10 00:54:23.792484 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:23.792431    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62x42\" (UniqueName: \"kubernetes.io/projected/77e24e29-3230-4a3c-b824-bc57866b2831-kube-api-access-62x42\") pod \"cert-manager-webhook-6486ffd997-xm956\" (UID: \"77e24e29-3230-4a3c-b824-bc57866b2831\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xm956"
Sep 10 00:54:23.910357 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:23.910312    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-xm956"
Sep 10 00:54:24.191701 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:24.191623    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-xm956"]
Sep 10 00:54:24.194607 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:54:24.194579    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod77e24e29_3230_4a3c_b824_bc57866b2831.slice/crio-45ee46ffd5ef88a001d40e74b26d7da98c4c8dea94b86838ecf53d73e73a961d WatchSource:0}: Error finding container 45ee46ffd5ef88a001d40e74b26d7da98c4c8dea94b86838ecf53d73e73a961d: Status 404 returned error can't find the container with id 45ee46ffd5ef88a001d40e74b26d7da98c4c8dea94b86838ecf53d73e73a961d
Sep 10 00:54:24.326610 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:24.326578    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-xm956" event={"ID":"77e24e29-3230-4a3c-b824-bc57866b2831","Type":"ContainerStarted","Data":"45ee46ffd5ef88a001d40e74b26d7da98c4c8dea94b86838ecf53d73e73a961d"}
Sep 10 00:54:28.346230 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:28.346193    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-xm956" event={"ID":"77e24e29-3230-4a3c-b824-bc57866b2831","Type":"ContainerStarted","Data":"d3f03e666182a0b0df94e310c951269cd96aa23ec105bee9f832de2d3b622a40"}
Sep 10 00:54:28.346620 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:28.346392    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-xm956"
Sep 10 00:54:28.367626 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:28.367553    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-xm956" podStartSLOduration=2.106969362 podStartE2EDuration="5.367531893s" totalImagesPullingTime="3.260562531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:54:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:54:28.36541867 +0000 UTC m=+284.862718249" watchObservedRunningTime="2026-09-10 00:54:28.367531893 +0000 UTC m=+284.864831472"
Sep 10 00:54:34.353092 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:34.353060    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-xm956"
Sep 10 00:54:43.933562 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:43.933532    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nnwxp_1cee7771-6381-4fad-8c1e-ad1098fe409b/console-operator/1.log"
Sep 10 00:54:43.936032 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:43.936005    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nnwxp_1cee7771-6381-4fad-8c1e-ad1098fe409b/console-operator/1.log"
Sep 10 00:54:43.945597 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:43.945576    2563 kubelet.go:1648] "Image garbage collection succeeded"
Sep 10 00:54:54.414867 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:54.414829    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5dc88f9465-bmqsl"]
Sep 10 00:54:54.418108 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:54.418086    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5dc88f9465-bmqsl"
Sep 10 00:54:54.421268 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:54.421246    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 10 00:54:54.421551 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:54.421531    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 10 00:54:54.421921 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:54.421900    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-8jt9s\""
Sep 10 00:54:54.421921 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:54.421906    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 10 00:54:54.422346 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:54.422331    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 10 00:54:54.442864 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:54.442843    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8c11b938-a977-4632-8f25-99f8645a495e-webhook-cert\") pod \"opendatahub-operator-controller-manager-5dc88f9465-bmqsl\" (UID: \"8c11b938-a977-4632-8f25-99f8645a495e\") " pod="opendatahub/opendatahub-operator-controller-manager-5dc88f9465-bmqsl"
Sep 10 00:54:54.442969 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:54.442956    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c11b938-a977-4632-8f25-99f8645a495e-tmp\") pod \"opendatahub-operator-controller-manager-5dc88f9465-bmqsl\" (UID: \"8c11b938-a977-4632-8f25-99f8645a495e\") " pod="opendatahub/opendatahub-operator-controller-manager-5dc88f9465-bmqsl"
Sep 10 00:54:54.443036 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:54.443001    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8c11b938-a977-4632-8f25-99f8645a495e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5dc88f9465-bmqsl\" (UID: \"8c11b938-a977-4632-8f25-99f8645a495e\") " pod="opendatahub/opendatahub-operator-controller-manager-5dc88f9465-bmqsl"
Sep 10 00:54:54.443036 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:54.443032    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8c11b938-a977-4632-8f25-99f8645a495e-manifests\") pod \"opendatahub-operator-controller-manager-5dc88f9465-bmqsl\" (UID: \"8c11b938-a977-4632-8f25-99f8645a495e\") " pod="opendatahub/opendatahub-operator-controller-manager-5dc88f9465-bmqsl"
Sep 10 00:54:54.443118 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:54.443064    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5df9k\" (UniqueName: \"kubernetes.io/projected/8c11b938-a977-4632-8f25-99f8645a495e-kube-api-access-5df9k\") pod \"opendatahub-operator-controller-manager-5dc88f9465-bmqsl\" (UID: \"8c11b938-a977-4632-8f25-99f8645a495e\") " pod="opendatahub/opendatahub-operator-controller-manager-5dc88f9465-bmqsl"
Sep 10 00:54:54.508896 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:54.508866    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5dc88f9465-bmqsl"]
Sep 10 00:54:54.544025 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:54.543977    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c11b938-a977-4632-8f25-99f8645a495e-tmp\") pod \"opendatahub-operator-controller-manager-5dc88f9465-bmqsl\" (UID: \"8c11b938-a977-4632-8f25-99f8645a495e\") " pod="opendatahub/opendatahub-operator-controller-manager-5dc88f9465-bmqsl"
Sep 10 00:54:54.544025 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:54.544023    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8c11b938-a977-4632-8f25-99f8645a495e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5dc88f9465-bmqsl\" (UID: \"8c11b938-a977-4632-8f25-99f8645a495e\") " pod="opendatahub/opendatahub-operator-controller-manager-5dc88f9465-bmqsl"
Sep 10 00:54:54.544206 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:54.544043    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8c11b938-a977-4632-8f25-99f8645a495e-manifests\") pod \"opendatahub-operator-controller-manager-5dc88f9465-bmqsl\" (UID: \"8c11b938-a977-4632-8f25-99f8645a495e\") " pod="opendatahub/opendatahub-operator-controller-manager-5dc88f9465-bmqsl"
Sep 10 00:54:54.544206 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:54.544075    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5df9k\" (UniqueName: \"kubernetes.io/projected/8c11b938-a977-4632-8f25-99f8645a495e-kube-api-access-5df9k\") pod \"opendatahub-operator-controller-manager-5dc88f9465-bmqsl\" (UID: \"8c11b938-a977-4632-8f25-99f8645a495e\") " pod="opendatahub/opendatahub-operator-controller-manager-5dc88f9465-bmqsl"
Sep 10 00:54:54.544206 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:54.544105    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8c11b938-a977-4632-8f25-99f8645a495e-webhook-cert\") pod \"opendatahub-operator-controller-manager-5dc88f9465-bmqsl\" (UID: \"8c11b938-a977-4632-8f25-99f8645a495e\") " pod="opendatahub/opendatahub-operator-controller-manager-5dc88f9465-bmqsl"
Sep 10 00:54:54.544491 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:54.544468    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8c11b938-a977-4632-8f25-99f8645a495e-manifests\") pod \"opendatahub-operator-controller-manager-5dc88f9465-bmqsl\" (UID: \"8c11b938-a977-4632-8f25-99f8645a495e\") " pod="opendatahub/opendatahub-operator-controller-manager-5dc88f9465-bmqsl"
Sep 10 00:54:54.546283 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:54.546247    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c11b938-a977-4632-8f25-99f8645a495e-tmp\") pod \"opendatahub-operator-controller-manager-5dc88f9465-bmqsl\" (UID: \"8c11b938-a977-4632-8f25-99f8645a495e\") " pod="opendatahub/opendatahub-operator-controller-manager-5dc88f9465-bmqsl"
Sep 10 00:54:54.546515 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:54.546494    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8c11b938-a977-4632-8f25-99f8645a495e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5dc88f9465-bmqsl\" (UID: \"8c11b938-a977-4632-8f25-99f8645a495e\") " pod="opendatahub/opendatahub-operator-controller-manager-5dc88f9465-bmqsl"
Sep 10 00:54:54.546599 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:54.546580    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8c11b938-a977-4632-8f25-99f8645a495e-webhook-cert\") pod \"opendatahub-operator-controller-manager-5dc88f9465-bmqsl\" (UID: \"8c11b938-a977-4632-8f25-99f8645a495e\") " pod="opendatahub/opendatahub-operator-controller-manager-5dc88f9465-bmqsl"
Sep 10 00:54:54.557868 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:54.557844    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5df9k\" (UniqueName: \"kubernetes.io/projected/8c11b938-a977-4632-8f25-99f8645a495e-kube-api-access-5df9k\") pod \"opendatahub-operator-controller-manager-5dc88f9465-bmqsl\" (UID: \"8c11b938-a977-4632-8f25-99f8645a495e\") " pod="opendatahub/opendatahub-operator-controller-manager-5dc88f9465-bmqsl"
Sep 10 00:54:54.727952 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:54.727861    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5dc88f9465-bmqsl"
Sep 10 00:54:55.016807 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:55.016736    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5dc88f9465-bmqsl"]
Sep 10 00:54:55.020053 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:54:55.020016    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c11b938_a977_4632_8f25_99f8645a495e.slice/crio-853594e3a7bf71eb010f604f1cf895e28ac6541992d4ad3697222b427f26754b WatchSource:0}: Error finding container 853594e3a7bf71eb010f604f1cf895e28ac6541992d4ad3697222b427f26754b: Status 404 returned error can't find the container with id 853594e3a7bf71eb010f604f1cf895e28ac6541992d4ad3697222b427f26754b
Sep 10 00:54:55.021818 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:55.021801    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 00:54:55.456558 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:55.456522    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5dc88f9465-bmqsl" event={"ID":"8c11b938-a977-4632-8f25-99f8645a495e","Type":"ContainerStarted","Data":"853594e3a7bf71eb010f604f1cf895e28ac6541992d4ad3697222b427f26754b"}
Sep 10 00:54:58.472861 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:58.472821    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5dc88f9465-bmqsl" event={"ID":"8c11b938-a977-4632-8f25-99f8645a495e","Type":"ContainerStarted","Data":"2bb4d13382f464893d2ce6e261d9e0eb634da9138bedef0384931a37eb5045e3"}
Sep 10 00:54:59.479240 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:59.479150    2563 generic.go:356] "Generic (PLEG): container finished" podID="8c11b938-a977-4632-8f25-99f8645a495e" containerID="2bb4d13382f464893d2ce6e261d9e0eb634da9138bedef0384931a37eb5045e3" exitCode=0
Sep 10 00:54:59.479585 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:54:59.479241    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5dc88f9465-bmqsl" event={"ID":"8c11b938-a977-4632-8f25-99f8645a495e","Type":"ContainerDied","Data":"2bb4d13382f464893d2ce6e261d9e0eb634da9138bedef0384931a37eb5045e3"}
Sep 10 00:55:00.485153 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:00.485120    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5dc88f9465-bmqsl" event={"ID":"8c11b938-a977-4632-8f25-99f8645a495e","Type":"ContainerStarted","Data":"add47f09cc0dd96a484547b42526a28feb15e27040ffb7c005925efce888520b"}
Sep 10 00:55:00.485526 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:00.485231    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5dc88f9465-bmqsl"
Sep 10 00:55:00.507375 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:00.507322    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5dc88f9465-bmqsl" podStartSLOduration=2.074048001 podStartE2EDuration="6.507305047s" totalImagesPullingTime="3.348507066s" totalInitContainerRuntime="1.08474998s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:54:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:55:00.505192527 +0000 UTC m=+317.002492109" watchObservedRunningTime="2026-09-10 00:55:00.507305047 +0000 UTC m=+317.004604626"
Sep 10 00:55:11.490806 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:11.490775    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5dc88f9465-bmqsl"
Sep 10 00:55:24.880158 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:24.880123    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-njqxx"]
Sep 10 00:55:24.883593 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:24.883574    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-njqxx"
Sep 10 00:55:24.886199 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:24.886174    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-zck6h\""
Sep 10 00:55:24.886288 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:24.886174    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 10 00:55:24.891701 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:24.891676    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-njqxx"]
Sep 10 00:55:25.000366 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:25.000331    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xzhd\" (UniqueName: \"kubernetes.io/projected/b0d433ff-4b9c-41cc-a06b-12fb4c96c6d7-kube-api-access-7xzhd\") pod \"odh-model-controller-7998c67dc4-njqxx\" (UID: \"b0d433ff-4b9c-41cc-a06b-12fb4c96c6d7\") " pod="opendatahub/odh-model-controller-7998c67dc4-njqxx"
Sep 10 00:55:25.000501 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:25.000418    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b0d433ff-4b9c-41cc-a06b-12fb4c96c6d7-cert\") pod \"odh-model-controller-7998c67dc4-njqxx\" (UID: \"b0d433ff-4b9c-41cc-a06b-12fb4c96c6d7\") " pod="opendatahub/odh-model-controller-7998c67dc4-njqxx"
Sep 10 00:55:25.101901 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:25.101849    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b0d433ff-4b9c-41cc-a06b-12fb4c96c6d7-cert\") pod \"odh-model-controller-7998c67dc4-njqxx\" (UID: \"b0d433ff-4b9c-41cc-a06b-12fb4c96c6d7\") " pod="opendatahub/odh-model-controller-7998c67dc4-njqxx"
Sep 10 00:55:25.102298 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:25.102250    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xzhd\" (UniqueName: \"kubernetes.io/projected/b0d433ff-4b9c-41cc-a06b-12fb4c96c6d7-kube-api-access-7xzhd\") pod \"odh-model-controller-7998c67dc4-njqxx\" (UID: \"b0d433ff-4b9c-41cc-a06b-12fb4c96c6d7\") " pod="opendatahub/odh-model-controller-7998c67dc4-njqxx"
Sep 10 00:55:25.104477 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:25.104455    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b0d433ff-4b9c-41cc-a06b-12fb4c96c6d7-cert\") pod \"odh-model-controller-7998c67dc4-njqxx\" (UID: \"b0d433ff-4b9c-41cc-a06b-12fb4c96c6d7\") " pod="opendatahub/odh-model-controller-7998c67dc4-njqxx"
Sep 10 00:55:25.110316 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:25.110295    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xzhd\" (UniqueName: \"kubernetes.io/projected/b0d433ff-4b9c-41cc-a06b-12fb4c96c6d7-kube-api-access-7xzhd\") pod \"odh-model-controller-7998c67dc4-njqxx\" (UID: \"b0d433ff-4b9c-41cc-a06b-12fb4c96c6d7\") " pod="opendatahub/odh-model-controller-7998c67dc4-njqxx"
Sep 10 00:55:25.197223 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:25.197132    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-njqxx"
Sep 10 00:55:25.437111 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:25.437079    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6nlmk"]
Sep 10 00:55:25.440362 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:25.440331    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-6nlmk"
Sep 10 00:55:25.444024 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:25.443998    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 10 00:55:25.455748 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:25.455722    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6nlmk"]
Sep 10 00:55:25.490907 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:25.490864    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-njqxx"]
Sep 10 00:55:25.495320 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:55:25.495285    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0d433ff_4b9c_41cc_a06b_12fb4c96c6d7.slice/crio-4d397b1b774581447e36811694d0bc710cc1da03a12b4563d5de341b3e40bd32 WatchSource:0}: Error finding container 4d397b1b774581447e36811694d0bc710cc1da03a12b4563d5de341b3e40bd32: Status 404 returned error can't find the container with id 4d397b1b774581447e36811694d0bc710cc1da03a12b4563d5de341b3e40bd32
Sep 10 00:55:25.594579 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:25.594542    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-njqxx" event={"ID":"b0d433ff-4b9c-41cc-a06b-12fb4c96c6d7","Type":"ContainerStarted","Data":"4d397b1b774581447e36811694d0bc710cc1da03a12b4563d5de341b3e40bd32"}
Sep 10 00:55:25.607052 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:25.607014    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdxnz\" (UniqueName: \"kubernetes.io/projected/deb73139-28d2-4ed7-bd29-d97a262c17a6-kube-api-access-sdxnz\") pod \"model-serving-api-54d89bd5fc-6nlmk\" (UID: \"deb73139-28d2-4ed7-bd29-d97a262c17a6\") " pod="opendatahub/model-serving-api-54d89bd5fc-6nlmk"
Sep 10 00:55:25.607200 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:25.607069    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/deb73139-28d2-4ed7-bd29-d97a262c17a6-tls-certs\") pod \"model-serving-api-54d89bd5fc-6nlmk\" (UID: \"deb73139-28d2-4ed7-bd29-d97a262c17a6\") " pod="opendatahub/model-serving-api-54d89bd5fc-6nlmk"
Sep 10 00:55:25.708427 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:25.708345    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/deb73139-28d2-4ed7-bd29-d97a262c17a6-tls-certs\") pod \"model-serving-api-54d89bd5fc-6nlmk\" (UID: \"deb73139-28d2-4ed7-bd29-d97a262c17a6\") " pod="opendatahub/model-serving-api-54d89bd5fc-6nlmk"
Sep 10 00:55:25.708574 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:25.708435    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdxnz\" (UniqueName: \"kubernetes.io/projected/deb73139-28d2-4ed7-bd29-d97a262c17a6-kube-api-access-sdxnz\") pod \"model-serving-api-54d89bd5fc-6nlmk\" (UID: \"deb73139-28d2-4ed7-bd29-d97a262c17a6\") " pod="opendatahub/model-serving-api-54d89bd5fc-6nlmk"
Sep 10 00:55:25.710883 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:25.710854    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/deb73139-28d2-4ed7-bd29-d97a262c17a6-tls-certs\") pod \"model-serving-api-54d89bd5fc-6nlmk\" (UID: \"deb73139-28d2-4ed7-bd29-d97a262c17a6\") " pod="opendatahub/model-serving-api-54d89bd5fc-6nlmk"
Sep 10 00:55:25.720710 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:25.720683    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdxnz\" (UniqueName: \"kubernetes.io/projected/deb73139-28d2-4ed7-bd29-d97a262c17a6-kube-api-access-sdxnz\") pod \"model-serving-api-54d89bd5fc-6nlmk\" (UID: \"deb73139-28d2-4ed7-bd29-d97a262c17a6\") " pod="opendatahub/model-serving-api-54d89bd5fc-6nlmk"
Sep 10 00:55:25.752424 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:25.752388    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-6nlmk"
Sep 10 00:55:26.031623 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:26.031593    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6nlmk"]
Sep 10 00:55:26.033374 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:55:26.033348    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddeb73139_28d2_4ed7_bd29_d97a262c17a6.slice/crio-1a2108ddaf2ba240eb3a1e7c945fff4bce5d67c7446712d221c1fa1fc8ff980b WatchSource:0}: Error finding container 1a2108ddaf2ba240eb3a1e7c945fff4bce5d67c7446712d221c1fa1fc8ff980b: Status 404 returned error can't find the container with id 1a2108ddaf2ba240eb3a1e7c945fff4bce5d67c7446712d221c1fa1fc8ff980b
Sep 10 00:55:26.601892 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:26.601847    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-6nlmk" event={"ID":"deb73139-28d2-4ed7-bd29-d97a262c17a6","Type":"ContainerStarted","Data":"1a2108ddaf2ba240eb3a1e7c945fff4bce5d67c7446712d221c1fa1fc8ff980b"}
Sep 10 00:55:30.624083 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:30.624041    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-njqxx" event={"ID":"b0d433ff-4b9c-41cc-a06b-12fb4c96c6d7","Type":"ContainerStarted","Data":"020eb039d40c708cc095e835faab7d35be2ee69e945f3d2a7b4e8a1f3d97afaf"}
Sep 10 00:55:30.624558 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:30.624201    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-njqxx"
Sep 10 00:55:30.625910 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:30.625883    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-6nlmk" event={"ID":"deb73139-28d2-4ed7-bd29-d97a262c17a6","Type":"ContainerStarted","Data":"46fab3b4b9c2781cb01ed04552256e21bdf966a077bde06d4d6ff5150b7c164c"}
Sep 10 00:55:30.626045 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:30.626034    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-6nlmk"
Sep 10 00:55:30.656782 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:30.656728    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-njqxx" podStartSLOduration=2.636209151 podStartE2EDuration="6.656712082s" totalImagesPullingTime="4.020502931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:55:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:55:30.654068543 +0000 UTC m=+347.151368122" watchObservedRunningTime="2026-09-10 00:55:30.656712082 +0000 UTC m=+347.154011660"
Sep 10 00:55:30.682282 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:30.682226    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-6nlmk" podStartSLOduration=2.15456791 podStartE2EDuration="5.682211391s" totalImagesPullingTime="3.527643481s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:55:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:55:30.681021515 +0000 UTC m=+347.178321092" watchObservedRunningTime="2026-09-10 00:55:30.682211391 +0000 UTC m=+347.179510969"
Sep 10 00:55:33.010790 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:33.010750    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-56bf56cd54-w6grw"]
Sep 10 00:55:33.019965 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:33.019940    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-56bf56cd54-w6grw"
Sep 10 00:55:33.024344 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:33.024323    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 10 00:55:33.025546 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:33.025522    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 10 00:55:33.025674 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:33.025583    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 10 00:55:33.025758 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:33.025688    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-qmdts\""
Sep 10 00:55:33.025817 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:33.025778    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 10 00:55:33.026193 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:33.026174    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 10 00:55:33.033071 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:33.033047    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-56bf56cd54-w6grw"]
Sep 10 00:55:33.180944 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:33.180904    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3a8425cd-4ccc-4117-8d29-b64f424da3be-cert\") pod \"lws-controller-manager-56bf56cd54-w6grw\" (UID: \"3a8425cd-4ccc-4117-8d29-b64f424da3be\") " pod="openshift-lws-operator/lws-controller-manager-56bf56cd54-w6grw"
Sep 10 00:55:33.181132 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:33.180966    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwmrg\" (UniqueName: \"kubernetes.io/projected/3a8425cd-4ccc-4117-8d29-b64f424da3be-kube-api-access-pwmrg\") pod \"lws-controller-manager-56bf56cd54-w6grw\" (UID: \"3a8425cd-4ccc-4117-8d29-b64f424da3be\") " pod="openshift-lws-operator/lws-controller-manager-56bf56cd54-w6grw"
Sep 10 00:55:33.181132 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:33.181068    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3a8425cd-4ccc-4117-8d29-b64f424da3be-metrics-cert\") pod \"lws-controller-manager-56bf56cd54-w6grw\" (UID: \"3a8425cd-4ccc-4117-8d29-b64f424da3be\") " pod="openshift-lws-operator/lws-controller-manager-56bf56cd54-w6grw"
Sep 10 00:55:33.181132 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:33.181097    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3a8425cd-4ccc-4117-8d29-b64f424da3be-manager-config\") pod \"lws-controller-manager-56bf56cd54-w6grw\" (UID: \"3a8425cd-4ccc-4117-8d29-b64f424da3be\") " pod="openshift-lws-operator/lws-controller-manager-56bf56cd54-w6grw"
Sep 10 00:55:33.281645 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:33.281567    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3a8425cd-4ccc-4117-8d29-b64f424da3be-cert\") pod \"lws-controller-manager-56bf56cd54-w6grw\" (UID: \"3a8425cd-4ccc-4117-8d29-b64f424da3be\") " pod="openshift-lws-operator/lws-controller-manager-56bf56cd54-w6grw"
Sep 10 00:55:33.281645 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:33.281610    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwmrg\" (UniqueName: \"kubernetes.io/projected/3a8425cd-4ccc-4117-8d29-b64f424da3be-kube-api-access-pwmrg\") pod \"lws-controller-manager-56bf56cd54-w6grw\" (UID: \"3a8425cd-4ccc-4117-8d29-b64f424da3be\") " pod="openshift-lws-operator/lws-controller-manager-56bf56cd54-w6grw"
Sep 10 00:55:33.281866 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:33.281648    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3a8425cd-4ccc-4117-8d29-b64f424da3be-metrics-cert\") pod \"lws-controller-manager-56bf56cd54-w6grw\" (UID: \"3a8425cd-4ccc-4117-8d29-b64f424da3be\") " pod="openshift-lws-operator/lws-controller-manager-56bf56cd54-w6grw"
Sep 10 00:55:33.281866 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:33.281663    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3a8425cd-4ccc-4117-8d29-b64f424da3be-manager-config\") pod \"lws-controller-manager-56bf56cd54-w6grw\" (UID: \"3a8425cd-4ccc-4117-8d29-b64f424da3be\") " pod="openshift-lws-operator/lws-controller-manager-56bf56cd54-w6grw"
Sep 10 00:55:33.282440 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:33.282415    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3a8425cd-4ccc-4117-8d29-b64f424da3be-manager-config\") pod \"lws-controller-manager-56bf56cd54-w6grw\" (UID: \"3a8425cd-4ccc-4117-8d29-b64f424da3be\") " pod="openshift-lws-operator/lws-controller-manager-56bf56cd54-w6grw"
Sep 10 00:55:33.284023 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:33.284002    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3a8425cd-4ccc-4117-8d29-b64f424da3be-cert\") pod \"lws-controller-manager-56bf56cd54-w6grw\" (UID: \"3a8425cd-4ccc-4117-8d29-b64f424da3be\") " pod="openshift-lws-operator/lws-controller-manager-56bf56cd54-w6grw"
Sep 10 00:55:33.284106 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:33.284042    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3a8425cd-4ccc-4117-8d29-b64f424da3be-metrics-cert\") pod \"lws-controller-manager-56bf56cd54-w6grw\" (UID: \"3a8425cd-4ccc-4117-8d29-b64f424da3be\") " pod="openshift-lws-operator/lws-controller-manager-56bf56cd54-w6grw"
Sep 10 00:55:33.290519 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:33.290495    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwmrg\" (UniqueName: \"kubernetes.io/projected/3a8425cd-4ccc-4117-8d29-b64f424da3be-kube-api-access-pwmrg\") pod \"lws-controller-manager-56bf56cd54-w6grw\" (UID: \"3a8425cd-4ccc-4117-8d29-b64f424da3be\") " pod="openshift-lws-operator/lws-controller-manager-56bf56cd54-w6grw"
Sep 10 00:55:33.330261 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:33.330223    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-56bf56cd54-w6grw"
Sep 10 00:55:33.612719 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:33.612691    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-56bf56cd54-w6grw"]
Sep 10 00:55:33.615276 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:55:33.615242    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a8425cd_4ccc_4117_8d29_b64f424da3be.slice/crio-91c4dccf53b250365918d80a94f357f7c7cd46d32a9cd2dac3981edc18cdc7e1 WatchSource:0}: Error finding container 91c4dccf53b250365918d80a94f357f7c7cd46d32a9cd2dac3981edc18cdc7e1: Status 404 returned error can't find the container with id 91c4dccf53b250365918d80a94f357f7c7cd46d32a9cd2dac3981edc18cdc7e1
Sep 10 00:55:33.640158 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:33.640124    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-56bf56cd54-w6grw" event={"ID":"3a8425cd-4ccc-4117-8d29-b64f424da3be","Type":"ContainerStarted","Data":"91c4dccf53b250365918d80a94f357f7c7cd46d32a9cd2dac3981edc18cdc7e1"}
Sep 10 00:55:34.082517 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:34.082440    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-m6mg5"]
Sep 10 00:55:34.087592 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:34.087573    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-m6mg5"
Sep 10 00:55:34.088807 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:34.088782    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mjzj\" (UniqueName: \"kubernetes.io/projected/3d3520e0-e82d-4178-9a58-07a44abccd49-kube-api-access-9mjzj\") pod \"kserve-controller-manager-d6f4b65b8-m6mg5\" (UID: \"3d3520e0-e82d-4178-9a58-07a44abccd49\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-m6mg5"
Sep 10 00:55:34.088944 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:34.088857    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3d3520e0-e82d-4178-9a58-07a44abccd49-cert\") pod \"kserve-controller-manager-d6f4b65b8-m6mg5\" (UID: \"3d3520e0-e82d-4178-9a58-07a44abccd49\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-m6mg5"
Sep 10 00:55:34.093136 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:34.093115    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-s5tc2\""
Sep 10 00:55:34.093303 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:34.093115    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 10 00:55:34.109662 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:34.109632    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-m6mg5"]
Sep 10 00:55:34.110965 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:34.110942    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-f7z4t"]
Sep 10 00:55:34.114935 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:34.114915    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f7z4t"
Sep 10 00:55:34.117885 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:34.117863    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 10 00:55:34.121843 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:34.121827    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-wk8rc\""
Sep 10 00:55:34.128564 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:34.128537    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-f7z4t"]
Sep 10 00:55:34.189320 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:34.189282    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3d3520e0-e82d-4178-9a58-07a44abccd49-cert\") pod \"kserve-controller-manager-d6f4b65b8-m6mg5\" (UID: \"3d3520e0-e82d-4178-9a58-07a44abccd49\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-m6mg5"
Sep 10 00:55:34.189515 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:34.189352    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mjzj\" (UniqueName: \"kubernetes.io/projected/3d3520e0-e82d-4178-9a58-07a44abccd49-kube-api-access-9mjzj\") pod \"kserve-controller-manager-d6f4b65b8-m6mg5\" (UID: \"3d3520e0-e82d-4178-9a58-07a44abccd49\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-m6mg5"
Sep 10 00:55:34.189515 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:55:34.189461    2563 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Sep 10 00:55:34.189647 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:55:34.189524    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d3520e0-e82d-4178-9a58-07a44abccd49-cert podName:3d3520e0-e82d-4178-9a58-07a44abccd49 nodeName:}" failed. No retries permitted until 2026-09-10 00:55:34.689504005 +0000 UTC m=+351.186803577 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/3d3520e0-e82d-4178-9a58-07a44abccd49-cert") pod "kserve-controller-manager-d6f4b65b8-m6mg5" (UID: "3d3520e0-e82d-4178-9a58-07a44abccd49") : secret "kserve-webhook-server-cert" not found
Sep 10 00:55:34.198815 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:34.198778    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mjzj\" (UniqueName: \"kubernetes.io/projected/3d3520e0-e82d-4178-9a58-07a44abccd49-kube-api-access-9mjzj\") pod \"kserve-controller-manager-d6f4b65b8-m6mg5\" (UID: \"3d3520e0-e82d-4178-9a58-07a44abccd49\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-m6mg5"
Sep 10 00:55:34.290525 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:34.290494    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stcww\" (UniqueName: \"kubernetes.io/projected/ed4f6ebe-d3d9-4014-8648-9536f3734cde-kube-api-access-stcww\") pod \"llmisvc-controller-manager-777c6bd7d5-f7z4t\" (UID: \"ed4f6ebe-d3d9-4014-8648-9536f3734cde\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f7z4t"
Sep 10 00:55:34.290675 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:34.290559    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ed4f6ebe-d3d9-4014-8648-9536f3734cde-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-f7z4t\" (UID: \"ed4f6ebe-d3d9-4014-8648-9536f3734cde\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f7z4t"
Sep 10 00:55:34.390964 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:34.390927    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ed4f6ebe-d3d9-4014-8648-9536f3734cde-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-f7z4t\" (UID: \"ed4f6ebe-d3d9-4014-8648-9536f3734cde\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f7z4t"
Sep 10 00:55:34.391187 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:34.391045    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stcww\" (UniqueName: \"kubernetes.io/projected/ed4f6ebe-d3d9-4014-8648-9536f3734cde-kube-api-access-stcww\") pod \"llmisvc-controller-manager-777c6bd7d5-f7z4t\" (UID: \"ed4f6ebe-d3d9-4014-8648-9536f3734cde\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f7z4t"
Sep 10 00:55:34.394138 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:34.394109    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ed4f6ebe-d3d9-4014-8648-9536f3734cde-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-f7z4t\" (UID: \"ed4f6ebe-d3d9-4014-8648-9536f3734cde\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f7z4t"
Sep 10 00:55:34.403083 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:34.403053    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stcww\" (UniqueName: \"kubernetes.io/projected/ed4f6ebe-d3d9-4014-8648-9536f3734cde-kube-api-access-stcww\") pod \"llmisvc-controller-manager-777c6bd7d5-f7z4t\" (UID: \"ed4f6ebe-d3d9-4014-8648-9536f3734cde\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f7z4t"
Sep 10 00:55:34.424923 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:34.424887    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f7z4t"
Sep 10 00:55:34.696081 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:34.695997    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3d3520e0-e82d-4178-9a58-07a44abccd49-cert\") pod \"kserve-controller-manager-d6f4b65b8-m6mg5\" (UID: \"3d3520e0-e82d-4178-9a58-07a44abccd49\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-m6mg5"
Sep 10 00:55:34.696301 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:55:34.696128    2563 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Sep 10 00:55:34.696301 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:55:34.696208    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d3520e0-e82d-4178-9a58-07a44abccd49-cert podName:3d3520e0-e82d-4178-9a58-07a44abccd49 nodeName:}" failed. No retries permitted until 2026-09-10 00:55:35.696186189 +0000 UTC m=+352.193485749 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/3d3520e0-e82d-4178-9a58-07a44abccd49-cert") pod "kserve-controller-manager-d6f4b65b8-m6mg5" (UID: "3d3520e0-e82d-4178-9a58-07a44abccd49") : secret "kserve-webhook-server-cert" not found
Sep 10 00:55:34.710962 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:34.710909    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-f7z4t"]
Sep 10 00:55:34.714073 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:55:34.714041    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poded4f6ebe_d3d9_4014_8648_9536f3734cde.slice/crio-6de57a84d64f0f5f2ab05a0ad9b994444bc2ab1be265a1b7c833aded6e33d6b7 WatchSource:0}: Error finding container 6de57a84d64f0f5f2ab05a0ad9b994444bc2ab1be265a1b7c833aded6e33d6b7: Status 404 returned error can't find the container with id 6de57a84d64f0f5f2ab05a0ad9b994444bc2ab1be265a1b7c833aded6e33d6b7
Sep 10 00:55:35.652665 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:35.652621    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f7z4t" event={"ID":"ed4f6ebe-d3d9-4014-8648-9536f3734cde","Type":"ContainerStarted","Data":"6de57a84d64f0f5f2ab05a0ad9b994444bc2ab1be265a1b7c833aded6e33d6b7"}
Sep 10 00:55:35.704816 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:35.704777    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3d3520e0-e82d-4178-9a58-07a44abccd49-cert\") pod \"kserve-controller-manager-d6f4b65b8-m6mg5\" (UID: \"3d3520e0-e82d-4178-9a58-07a44abccd49\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-m6mg5"
Sep 10 00:55:35.707370 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:35.707335    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3d3520e0-e82d-4178-9a58-07a44abccd49-cert\") pod \"kserve-controller-manager-d6f4b65b8-m6mg5\" (UID: \"3d3520e0-e82d-4178-9a58-07a44abccd49\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-m6mg5"
Sep 10 00:55:35.898906 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:35.898867    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-m6mg5"
Sep 10 00:55:36.570579 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:36.570551    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-m6mg5"]
Sep 10 00:55:36.644374 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:55:36.644335    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d3520e0_e82d_4178_9a58_07a44abccd49.slice/crio-8b390e282bf425becdfc6e191d252c66e1f7588de89001a20ed7e63144e4412d WatchSource:0}: Error finding container 8b390e282bf425becdfc6e191d252c66e1f7588de89001a20ed7e63144e4412d: Status 404 returned error can't find the container with id 8b390e282bf425becdfc6e191d252c66e1f7588de89001a20ed7e63144e4412d
Sep 10 00:55:36.659307 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:36.659273    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-m6mg5" event={"ID":"3d3520e0-e82d-4178-9a58-07a44abccd49","Type":"ContainerStarted","Data":"8b390e282bf425becdfc6e191d252c66e1f7588de89001a20ed7e63144e4412d"}
Sep 10 00:55:37.666635 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:37.666594    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-56bf56cd54-w6grw" event={"ID":"3a8425cd-4ccc-4117-8d29-b64f424da3be","Type":"ContainerStarted","Data":"f3395d5a5f2c034c2a2799edbb1d52ac70964bc706971ad847eff9dba275e35d"}
Sep 10 00:55:37.667079 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:37.666750    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-56bf56cd54-w6grw"
Sep 10 00:55:37.684476 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:37.684430    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-56bf56cd54-w6grw" podStartSLOduration=2.765647972 podStartE2EDuration="5.684414492s" totalImagesPullingTime="2.91876652s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:55:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:55:37.682013444 +0000 UTC m=+354.179313024" watchObservedRunningTime="2026-09-10 00:55:37.684414492 +0000 UTC m=+354.181714133"
Sep 10 00:55:39.680962 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:39.680853    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-m6mg5" event={"ID":"3d3520e0-e82d-4178-9a58-07a44abccd49","Type":"ContainerStarted","Data":"65cccc6c70a87110b90274308d3ad4ea2b566c57bfad3fa68416a6a205d219a8"}
Sep 10 00:55:39.681422 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:39.681031    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-m6mg5"
Sep 10 00:55:39.682898 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:39.682871    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f7z4t" event={"ID":"ed4f6ebe-d3d9-4014-8648-9536f3734cde","Type":"ContainerStarted","Data":"e9f6ece453b18002c5cfc30767ed6c9d7e8292ec42194d6e26bbeb8667097484"}
Sep 10 00:55:39.683059 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:39.683038    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f7z4t"
Sep 10 00:55:39.696261 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:39.696220    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-m6mg5" podStartSLOduration=2.953134713 podStartE2EDuration="5.696196192s" totalImagesPullingTime="2.743061479s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:55:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:55:39.695235806 +0000 UTC m=+356.192535414" watchObservedRunningTime="2026-09-10 00:55:39.696196192 +0000 UTC m=+356.193495771"
Sep 10 00:55:39.711226 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:39.711179    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f7z4t" podStartSLOduration=1.039904839 podStartE2EDuration="5.711165029s" totalImagesPullingTime="4.67126019s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:55:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:55:39.710014038 +0000 UTC m=+356.207313617" watchObservedRunningTime="2026-09-10 00:55:39.711165029 +0000 UTC m=+356.208464637"
Sep 10 00:55:41.632661 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:41.632623    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-njqxx"
Sep 10 00:55:41.634503 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:41.634480    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-6nlmk"
Sep 10 00:55:48.677335 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:55:48.677305    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-56bf56cd54-w6grw"
Sep 10 00:56:10.690661 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:10.690627    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f7z4t"
Sep 10 00:56:10.693492 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:10.693468    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-m6mg5"
Sep 10 00:56:34.108136 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:34.108099    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5fd968dd46-n9nq6"]
Sep 10 00:56:34.116423 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:34.116397    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fd968dd46-n9nq6"
Sep 10 00:56:34.120073 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:34.120048    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 10 00:56:34.120211 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:34.120050    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 10 00:56:34.121490 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:34.121468    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 10 00:56:34.121613 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:34.121509    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 10 00:56:34.121613 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:34.121517    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 10 00:56:34.121613 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:34.121590    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jlrnr\""
Sep 10 00:56:34.121613 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:34.121512    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 10 00:56:34.123088 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:34.123058    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 10 00:56:34.126923 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:34.126901    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5fd968dd46-n9nq6"]
Sep 10 00:56:34.127397 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:34.127376    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 10 00:56:34.224784 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:34.224751    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6613e142-5fea-4fc4-a420-1efccaf5d344-console-serving-cert\") pod \"console-5fd968dd46-n9nq6\" (UID: \"6613e142-5fea-4fc4-a420-1efccaf5d344\") " pod="openshift-console/console-5fd968dd46-n9nq6"
Sep 10 00:56:34.224784 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:34.224786    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2c2d\" (UniqueName: \"kubernetes.io/projected/6613e142-5fea-4fc4-a420-1efccaf5d344-kube-api-access-r2c2d\") pod \"console-5fd968dd46-n9nq6\" (UID: \"6613e142-5fea-4fc4-a420-1efccaf5d344\") " pod="openshift-console/console-5fd968dd46-n9nq6"
Sep 10 00:56:34.224979 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:34.224808    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6613e142-5fea-4fc4-a420-1efccaf5d344-oauth-serving-cert\") pod \"console-5fd968dd46-n9nq6\" (UID: \"6613e142-5fea-4fc4-a420-1efccaf5d344\") " pod="openshift-console/console-5fd968dd46-n9nq6"
Sep 10 00:56:34.224979 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:34.224825    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6613e142-5fea-4fc4-a420-1efccaf5d344-console-oauth-config\") pod \"console-5fd968dd46-n9nq6\" (UID: \"6613e142-5fea-4fc4-a420-1efccaf5d344\") " pod="openshift-console/console-5fd968dd46-n9nq6"
Sep 10 00:56:34.224979 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:34.224908    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6613e142-5fea-4fc4-a420-1efccaf5d344-trusted-ca-bundle\") pod \"console-5fd968dd46-n9nq6\" (UID: \"6613e142-5fea-4fc4-a420-1efccaf5d344\") " pod="openshift-console/console-5fd968dd46-n9nq6"
Sep 10 00:56:34.224979 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:34.224948    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6613e142-5fea-4fc4-a420-1efccaf5d344-service-ca\") pod \"console-5fd968dd46-n9nq6\" (UID: \"6613e142-5fea-4fc4-a420-1efccaf5d344\") " pod="openshift-console/console-5fd968dd46-n9nq6"
Sep 10 00:56:34.224979 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:34.224977    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6613e142-5fea-4fc4-a420-1efccaf5d344-console-config\") pod \"console-5fd968dd46-n9nq6\" (UID: \"6613e142-5fea-4fc4-a420-1efccaf5d344\") " pod="openshift-console/console-5fd968dd46-n9nq6"
Sep 10 00:56:34.326376 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:34.326334    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6613e142-5fea-4fc4-a420-1efccaf5d344-oauth-serving-cert\") pod \"console-5fd968dd46-n9nq6\" (UID: \"6613e142-5fea-4fc4-a420-1efccaf5d344\") " pod="openshift-console/console-5fd968dd46-n9nq6"
Sep 10 00:56:34.326376 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:34.326380    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6613e142-5fea-4fc4-a420-1efccaf5d344-console-oauth-config\") pod \"console-5fd968dd46-n9nq6\" (UID: \"6613e142-5fea-4fc4-a420-1efccaf5d344\") " pod="openshift-console/console-5fd968dd46-n9nq6"
Sep 10 00:56:34.326636 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:34.326414    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6613e142-5fea-4fc4-a420-1efccaf5d344-trusted-ca-bundle\") pod \"console-5fd968dd46-n9nq6\" (UID: \"6613e142-5fea-4fc4-a420-1efccaf5d344\") " pod="openshift-console/console-5fd968dd46-n9nq6"
Sep 10 00:56:34.326636 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:34.326450    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6613e142-5fea-4fc4-a420-1efccaf5d344-service-ca\") pod \"console-5fd968dd46-n9nq6\" (UID: \"6613e142-5fea-4fc4-a420-1efccaf5d344\") " pod="openshift-console/console-5fd968dd46-n9nq6"
Sep 10 00:56:34.326636 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:34.326487    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6613e142-5fea-4fc4-a420-1efccaf5d344-console-config\") pod \"console-5fd968dd46-n9nq6\" (UID: \"6613e142-5fea-4fc4-a420-1efccaf5d344\") " pod="openshift-console/console-5fd968dd46-n9nq6"
Sep 10 00:56:34.326636 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:34.326555    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6613e142-5fea-4fc4-a420-1efccaf5d344-console-serving-cert\") pod \"console-5fd968dd46-n9nq6\" (UID: \"6613e142-5fea-4fc4-a420-1efccaf5d344\") " pod="openshift-console/console-5fd968dd46-n9nq6"
Sep 10 00:56:34.326636 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:34.326583    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2c2d\" (UniqueName: \"kubernetes.io/projected/6613e142-5fea-4fc4-a420-1efccaf5d344-kube-api-access-r2c2d\") pod \"console-5fd968dd46-n9nq6\" (UID: \"6613e142-5fea-4fc4-a420-1efccaf5d344\") " pod="openshift-console/console-5fd968dd46-n9nq6"
Sep 10 00:56:34.327230 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:34.327200    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6613e142-5fea-4fc4-a420-1efccaf5d344-service-ca\") pod \"console-5fd968dd46-n9nq6\" (UID: \"6613e142-5fea-4fc4-a420-1efccaf5d344\") " pod="openshift-console/console-5fd968dd46-n9nq6"
Sep 10 00:56:34.327356 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:34.327250    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6613e142-5fea-4fc4-a420-1efccaf5d344-oauth-serving-cert\") pod \"console-5fd968dd46-n9nq6\" (UID: \"6613e142-5fea-4fc4-a420-1efccaf5d344\") " pod="openshift-console/console-5fd968dd46-n9nq6"
Sep 10 00:56:34.327356 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:34.327337    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6613e142-5fea-4fc4-a420-1efccaf5d344-console-config\") pod \"console-5fd968dd46-n9nq6\" (UID: \"6613e142-5fea-4fc4-a420-1efccaf5d344\") " pod="openshift-console/console-5fd968dd46-n9nq6"
Sep 10 00:56:34.327473 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:34.327392    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6613e142-5fea-4fc4-a420-1efccaf5d344-trusted-ca-bundle\") pod \"console-5fd968dd46-n9nq6\" (UID: \"6613e142-5fea-4fc4-a420-1efccaf5d344\") " pod="openshift-console/console-5fd968dd46-n9nq6"
Sep 10 00:56:34.328939 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:34.328919    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6613e142-5fea-4fc4-a420-1efccaf5d344-console-oauth-config\") pod \"console-5fd968dd46-n9nq6\" (UID: \"6613e142-5fea-4fc4-a420-1efccaf5d344\") " pod="openshift-console/console-5fd968dd46-n9nq6"
Sep 10 00:56:34.329181 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:34.329162    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6613e142-5fea-4fc4-a420-1efccaf5d344-console-serving-cert\") pod \"console-5fd968dd46-n9nq6\" (UID: \"6613e142-5fea-4fc4-a420-1efccaf5d344\") " pod="openshift-console/console-5fd968dd46-n9nq6"
Sep 10 00:56:34.334888 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:34.334861    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2c2d\" (UniqueName: \"kubernetes.io/projected/6613e142-5fea-4fc4-a420-1efccaf5d344-kube-api-access-r2c2d\") pod \"console-5fd968dd46-n9nq6\" (UID: \"6613e142-5fea-4fc4-a420-1efccaf5d344\") " pod="openshift-console/console-5fd968dd46-n9nq6"
Sep 10 00:56:34.427837 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:34.427752    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fd968dd46-n9nq6"
Sep 10 00:56:34.703233 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:34.703209    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5fd968dd46-n9nq6"]
Sep 10 00:56:34.705589 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:56:34.705557    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6613e142_5fea_4fc4_a420_1efccaf5d344.slice/crio-97ee189036414924b7a35ee9089365e5841da9ac654b7226d5205d1fa5aba720 WatchSource:0}: Error finding container 97ee189036414924b7a35ee9089365e5841da9ac654b7226d5205d1fa5aba720: Status 404 returned error can't find the container with id 97ee189036414924b7a35ee9089365e5841da9ac654b7226d5205d1fa5aba720
Sep 10 00:56:34.930025 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:34.929969    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fd968dd46-n9nq6" event={"ID":"6613e142-5fea-4fc4-a420-1efccaf5d344","Type":"ContainerStarted","Data":"9872f41494d61fe485b63788893121c22a4698488eaea5cc9e2cd4693e486e2a"}
Sep 10 00:56:34.930193 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:34.930037    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fd968dd46-n9nq6" event={"ID":"6613e142-5fea-4fc4-a420-1efccaf5d344","Type":"ContainerStarted","Data":"97ee189036414924b7a35ee9089365e5841da9ac654b7226d5205d1fa5aba720"}
Sep 10 00:56:34.948248 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:34.948196    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5fd968dd46-n9nq6" podStartSLOduration=0.948178987 podStartE2EDuration="948.178987ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:56:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:56:34.946933304 +0000 UTC m=+411.444232883" watchObservedRunningTime="2026-09-10 00:56:34.948178987 +0000 UTC m=+411.445478566"
Sep 10 00:56:44.428601 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:44.428560    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5fd968dd46-n9nq6"
Sep 10 00:56:44.429102 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:44.428614    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5fd968dd46-n9nq6"
Sep 10 00:56:44.433239 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:44.433218    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5fd968dd46-n9nq6"
Sep 10 00:56:44.979582 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:44.979548    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5fd968dd46-n9nq6"
Sep 10 00:56:54.564643 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:54.564604    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5vh2f"]
Sep 10 00:56:54.568488 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:54.568463    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5vh2f"
Sep 10 00:56:54.571869 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:54.571846    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 00:56:54.571997 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:54.571912    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 00:56:54.571997 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:54.571929    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 10 00:56:54.572955 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:54.572940    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-vmwp8\""
Sep 10 00:56:54.580501 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:54.580479    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5vh2f"]
Sep 10 00:56:54.708840 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:54.708789    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs967\" (UniqueName: \"kubernetes.io/projected/1c8597bd-5ba5-4fc5-ab46-f7d7d8bedd0f-kube-api-access-bs967\") pod \"dns-operator-controller-manager-7b84cd7c9c-5vh2f\" (UID: \"1c8597bd-5ba5-4fc5-ab46-f7d7d8bedd0f\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5vh2f"
Sep 10 00:56:54.809951 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:54.809900    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bs967\" (UniqueName: \"kubernetes.io/projected/1c8597bd-5ba5-4fc5-ab46-f7d7d8bedd0f-kube-api-access-bs967\") pod \"dns-operator-controller-manager-7b84cd7c9c-5vh2f\" (UID: \"1c8597bd-5ba5-4fc5-ab46-f7d7d8bedd0f\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5vh2f"
Sep 10 00:56:54.822821 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:54.822766    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs967\" (UniqueName: \"kubernetes.io/projected/1c8597bd-5ba5-4fc5-ab46-f7d7d8bedd0f-kube-api-access-bs967\") pod \"dns-operator-controller-manager-7b84cd7c9c-5vh2f\" (UID: \"1c8597bd-5ba5-4fc5-ab46-f7d7d8bedd0f\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5vh2f"
Sep 10 00:56:54.878697 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:54.878662    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5vh2f"
Sep 10 00:56:55.148162 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:55.148138    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5vh2f"]
Sep 10 00:56:55.150855 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:56:55.150819    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c8597bd_5ba5_4fc5_ab46_f7d7d8bedd0f.slice/crio-c09dee7f1956e19b1cfca50dffd270534167737ff0fed42b2641bcf7c415298b WatchSource:0}: Error finding container c09dee7f1956e19b1cfca50dffd270534167737ff0fed42b2641bcf7c415298b: Status 404 returned error can't find the container with id c09dee7f1956e19b1cfca50dffd270534167737ff0fed42b2641bcf7c415298b
Sep 10 00:56:56.027968 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:56.027926    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5vh2f" event={"ID":"1c8597bd-5ba5-4fc5-ab46-f7d7d8bedd0f","Type":"ContainerStarted","Data":"c09dee7f1956e19b1cfca50dffd270534167737ff0fed42b2641bcf7c415298b"}
Sep 10 00:56:57.326939 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:57.326904    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-m979s"]
Sep 10 00:56:57.331000 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:57.330963    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-m979s"
Sep 10 00:56:57.334153 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:57.334131    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-g5n77\""
Sep 10 00:56:57.346590 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:57.346563    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-m979s"]
Sep 10 00:56:57.434935 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:57.434904    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjd7q\" (UniqueName: \"kubernetes.io/projected/acfc4eb4-0449-468b-88cf-acfe8976858b-kube-api-access-tjd7q\") pod \"authorino-operator-76b478bd78-m979s\" (UID: \"acfc4eb4-0449-468b-88cf-acfe8976858b\") " pod="kuadrant-system/authorino-operator-76b478bd78-m979s"
Sep 10 00:56:57.535996 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:57.535945    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjd7q\" (UniqueName: \"kubernetes.io/projected/acfc4eb4-0449-468b-88cf-acfe8976858b-kube-api-access-tjd7q\") pod \"authorino-operator-76b478bd78-m979s\" (UID: \"acfc4eb4-0449-468b-88cf-acfe8976858b\") " pod="kuadrant-system/authorino-operator-76b478bd78-m979s"
Sep 10 00:56:57.552472 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:57.552441    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjd7q\" (UniqueName: \"kubernetes.io/projected/acfc4eb4-0449-468b-88cf-acfe8976858b-kube-api-access-tjd7q\") pod \"authorino-operator-76b478bd78-m979s\" (UID: \"acfc4eb4-0449-468b-88cf-acfe8976858b\") " pod="kuadrant-system/authorino-operator-76b478bd78-m979s"
Sep 10 00:56:57.642621 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:57.642595    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-m979s"
Sep 10 00:56:57.920765 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:57.920742    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-m979s"]
Sep 10 00:56:57.922471 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:56:57.922442    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacfc4eb4_0449_468b_88cf_acfe8976858b.slice/crio-b631e1749de70491151470dc99d1266364c3968a1b2d7ef84d0d821362822f11 WatchSource:0}: Error finding container b631e1749de70491151470dc99d1266364c3968a1b2d7ef84d0d821362822f11: Status 404 returned error can't find the container with id b631e1749de70491151470dc99d1266364c3968a1b2d7ef84d0d821362822f11
Sep 10 00:56:58.043632 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:58.043596    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-m979s" event={"ID":"acfc4eb4-0449-468b-88cf-acfe8976858b","Type":"ContainerStarted","Data":"b631e1749de70491151470dc99d1266364c3968a1b2d7ef84d0d821362822f11"}
Sep 10 00:56:58.045704 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:58.045667    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5vh2f" event={"ID":"1c8597bd-5ba5-4fc5-ab46-f7d7d8bedd0f","Type":"ContainerStarted","Data":"0db52781dde511d486643f6484d004bac7c09807d4947dc4d6bd7313091fed0a"}
Sep 10 00:56:58.045856 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:58.045838    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5vh2f"
Sep 10 00:56:58.073419 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:58.073358    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5vh2f" podStartSLOduration=1.621064799 podStartE2EDuration="4.073340104s" totalImagesPullingTime="2.452275305s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:56:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:56:58.072254196 +0000 UTC m=+434.569553776" watchObservedRunningTime="2026-09-10 00:56:58.073340104 +0000 UTC m=+434.570639685"
Sep 10 00:56:59.502082 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:59.502043    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-27f57"]
Sep 10 00:56:59.505508 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:59.505490    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-27f57"
Sep 10 00:56:59.507909 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:59.507881    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-fsqkd\""
Sep 10 00:56:59.515922 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:59.515895    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-27f57"]
Sep 10 00:56:59.658033 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:59.657925    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkmbd\" (UniqueName: \"kubernetes.io/projected/d98235fe-14bd-497c-be43-6f124b60ccdd-kube-api-access-vkmbd\") pod \"limitador-operator-controller-manager-74495c69f8-27f57\" (UID: \"d98235fe-14bd-497c-be43-6f124b60ccdd\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-27f57"
Sep 10 00:56:59.758885 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:59.758842    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkmbd\" (UniqueName: \"kubernetes.io/projected/d98235fe-14bd-497c-be43-6f124b60ccdd-kube-api-access-vkmbd\") pod \"limitador-operator-controller-manager-74495c69f8-27f57\" (UID: \"d98235fe-14bd-497c-be43-6f124b60ccdd\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-27f57"
Sep 10 00:56:59.767013 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:59.766959    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkmbd\" (UniqueName: \"kubernetes.io/projected/d98235fe-14bd-497c-be43-6f124b60ccdd-kube-api-access-vkmbd\") pod \"limitador-operator-controller-manager-74495c69f8-27f57\" (UID: \"d98235fe-14bd-497c-be43-6f124b60ccdd\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-27f57"
Sep 10 00:56:59.815770 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:56:59.815730    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-27f57"
Sep 10 00:57:00.056903 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:00.056808    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-m979s" event={"ID":"acfc4eb4-0449-468b-88cf-acfe8976858b","Type":"ContainerStarted","Data":"445758ae6f72646293b538e12a4d90a13362ed2c581361c38436f0b0a4e32a7d"}
Sep 10 00:57:00.057078 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:00.056968    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-m979s"
Sep 10 00:57:00.073708 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:00.073656    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-m979s" podStartSLOduration=1.614267628 podStartE2EDuration="3.073636288s" totalImagesPullingTime="1.45936866s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:56:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:57:00.07215693 +0000 UTC m=+436.569456508" watchObservedRunningTime="2026-09-10 00:57:00.073636288 +0000 UTC m=+436.570935867"
Sep 10 00:57:00.088559 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:00.088536    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-27f57"]
Sep 10 00:57:00.091201 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:57:00.091168    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd98235fe_14bd_497c_be43_6f124b60ccdd.slice/crio-55d24aa316aeeed48fb45b14b2ec89a14fef6aa456390113682e3f204fb0d150 WatchSource:0}: Error finding container 55d24aa316aeeed48fb45b14b2ec89a14fef6aa456390113682e3f204fb0d150: Status 404 returned error can't find the container with id 55d24aa316aeeed48fb45b14b2ec89a14fef6aa456390113682e3f204fb0d150
Sep 10 00:57:01.063189 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:01.063158    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-27f57" event={"ID":"d98235fe-14bd-497c-be43-6f124b60ccdd","Type":"ContainerStarted","Data":"55d24aa316aeeed48fb45b14b2ec89a14fef6aa456390113682e3f204fb0d150"}
Sep 10 00:57:03.074176 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:03.074137    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-27f57" event={"ID":"d98235fe-14bd-497c-be43-6f124b60ccdd","Type":"ContainerStarted","Data":"f4a48001b8bf70c5a61451fed73e3811d5f16f761dd09139e2831940916b52e9"}
Sep 10 00:57:03.074572 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:03.074210    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-27f57"
Sep 10 00:57:03.091965 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:03.091912    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-27f57" podStartSLOduration=1.704433393 podStartE2EDuration="4.091896949s" totalImagesPullingTime="2.387463556s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:56:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:57:03.089754558 +0000 UTC m=+439.587054138" watchObservedRunningTime="2026-09-10 00:57:03.091896949 +0000 UTC m=+439.589196528"
Sep 10 00:57:09.053702 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:09.053625    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5vh2f"
Sep 10 00:57:11.069356 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:11.069322    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-m979s"
Sep 10 00:57:12.037850 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:12.037809    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-27f57"]
Sep 10 00:57:12.038143 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:12.038115    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-27f57" podUID="d98235fe-14bd-497c-be43-6f124b60ccdd" containerName="manager" containerID="cri-o://f4a48001b8bf70c5a61451fed73e3811d5f16f761dd09139e2831940916b52e9" gracePeriod=2
Sep 10 00:57:12.040074 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:12.040047    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-27f57"
Sep 10 00:57:12.061758 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:12.061731    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-27f57"]
Sep 10 00:57:12.085511 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:57:12.085453    2563 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"d98235fe-14bd-497c-be43-6f124b60ccdd\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-74495c69f8-27f57\": pods \"limitador-operator-controller-manager-74495c69f8-27f57\" not found" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-27f57"
Sep 10 00:57:12.088152 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:12.088126    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-rzw4v"]
Sep 10 00:57:12.088519 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:12.088506    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d98235fe-14bd-497c-be43-6f124b60ccdd" containerName="manager"
Sep 10 00:57:12.088569 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:12.088521    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d98235fe-14bd-497c-be43-6f124b60ccdd" containerName="manager"
Sep 10 00:57:12.088605 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:12.088595    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d98235fe-14bd-497c-be43-6f124b60ccdd" containerName="manager"
Sep 10 00:57:12.091663 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:12.091644    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rzw4v"
Sep 10 00:57:12.120076 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:12.120048    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-rzw4v"]
Sep 10 00:57:12.159101 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:12.159065    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kq9vd\" (UniqueName: \"kubernetes.io/projected/f9f4d856-cc01-4df4-bf3f-f103bea1d309-kube-api-access-kq9vd\") pod \"limitador-operator-controller-manager-74495c69f8-rzw4v\" (UID: \"f9f4d856-cc01-4df4-bf3f-f103bea1d309\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rzw4v"
Sep 10 00:57:12.259733 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:12.259698    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kq9vd\" (UniqueName: \"kubernetes.io/projected/f9f4d856-cc01-4df4-bf3f-f103bea1d309-kube-api-access-kq9vd\") pod \"limitador-operator-controller-manager-74495c69f8-rzw4v\" (UID: \"f9f4d856-cc01-4df4-bf3f-f103bea1d309\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rzw4v"
Sep 10 00:57:12.268630 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:12.268607    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kq9vd\" (UniqueName: \"kubernetes.io/projected/f9f4d856-cc01-4df4-bf3f-f103bea1d309-kube-api-access-kq9vd\") pod \"limitador-operator-controller-manager-74495c69f8-rzw4v\" (UID: \"f9f4d856-cc01-4df4-bf3f-f103bea1d309\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rzw4v"
Sep 10 00:57:12.286834 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:12.286807    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-27f57"
Sep 10 00:57:12.360359 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:12.360322    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vkmbd\" (UniqueName: \"kubernetes.io/projected/d98235fe-14bd-497c-be43-6f124b60ccdd-kube-api-access-vkmbd\") pod \"d98235fe-14bd-497c-be43-6f124b60ccdd\" (UID: \"d98235fe-14bd-497c-be43-6f124b60ccdd\") "
Sep 10 00:57:12.362413 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:12.362388    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d98235fe-14bd-497c-be43-6f124b60ccdd-kube-api-access-vkmbd" (OuterVolumeSpecName: "kube-api-access-vkmbd") pod "d98235fe-14bd-497c-be43-6f124b60ccdd" (UID: "d98235fe-14bd-497c-be43-6f124b60ccdd"). InnerVolumeSpecName "kube-api-access-vkmbd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 00:57:12.421263 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:12.421221    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rzw4v"
Sep 10 00:57:12.461659 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:12.461614    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vkmbd\" (UniqueName: \"kubernetes.io/projected/d98235fe-14bd-497c-be43-6f124b60ccdd-kube-api-access-vkmbd\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:57:12.697721 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:12.697694    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-rzw4v"]
Sep 10 00:57:12.700188 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:57:12.700161    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9f4d856_cc01_4df4_bf3f_f103bea1d309.slice/crio-a15850b79a900a33fe34f87d4e5dfe5ceb6e7a25915968a92b483008267ac0ec WatchSource:0}: Error finding container a15850b79a900a33fe34f87d4e5dfe5ceb6e7a25915968a92b483008267ac0ec: Status 404 returned error can't find the container with id a15850b79a900a33fe34f87d4e5dfe5ceb6e7a25915968a92b483008267ac0ec
Sep 10 00:57:13.123117 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:13.123081    2563 generic.go:356] "Generic (PLEG): container finished" podID="d98235fe-14bd-497c-be43-6f124b60ccdd" containerID="f4a48001b8bf70c5a61451fed73e3811d5f16f761dd09139e2831940916b52e9" exitCode=0
Sep 10 00:57:13.123503 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:13.123126    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-27f57"
Sep 10 00:57:13.123503 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:13.123181    2563 scope.go:117] "RemoveContainer" containerID="f4a48001b8bf70c5a61451fed73e3811d5f16f761dd09139e2831940916b52e9"
Sep 10 00:57:13.125056 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:13.125024    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rzw4v" event={"ID":"f9f4d856-cc01-4df4-bf3f-f103bea1d309","Type":"ContainerStarted","Data":"6e56658bb8377ee7ce7453d6303b26e0b4e8deb42d19ab4de018eaefadf318dd"}
Sep 10 00:57:13.125192 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:13.125079    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rzw4v" event={"ID":"f9f4d856-cc01-4df4-bf3f-f103bea1d309","Type":"ContainerStarted","Data":"a15850b79a900a33fe34f87d4e5dfe5ceb6e7a25915968a92b483008267ac0ec"}
Sep 10 00:57:13.125192 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:13.125133    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rzw4v"
Sep 10 00:57:13.131998 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:13.131970    2563 scope.go:117] "RemoveContainer" containerID="f4a48001b8bf70c5a61451fed73e3811d5f16f761dd09139e2831940916b52e9"
Sep 10 00:57:13.132252 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:57:13.132234    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4a48001b8bf70c5a61451fed73e3811d5f16f761dd09139e2831940916b52e9\": container with ID starting with f4a48001b8bf70c5a61451fed73e3811d5f16f761dd09139e2831940916b52e9 not found: ID does not exist" containerID="f4a48001b8bf70c5a61451fed73e3811d5f16f761dd09139e2831940916b52e9"
Sep 10 00:57:13.132295 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:13.132260    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4a48001b8bf70c5a61451fed73e3811d5f16f761dd09139e2831940916b52e9"} err="failed to get container status \"f4a48001b8bf70c5a61451fed73e3811d5f16f761dd09139e2831940916b52e9\": rpc error: code = NotFound desc = could not find container \"f4a48001b8bf70c5a61451fed73e3811d5f16f761dd09139e2831940916b52e9\": container with ID starting with f4a48001b8bf70c5a61451fed73e3811d5f16f761dd09139e2831940916b52e9 not found: ID does not exist"
Sep 10 00:57:13.142319 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:13.142279    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rzw4v" podStartSLOduration=1.14226735 podStartE2EDuration="1.14226735s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:57:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:57:13.141315281 +0000 UTC m=+449.638614861" watchObservedRunningTime="2026-09-10 00:57:13.14226735 +0000 UTC m=+449.639566928"
Sep 10 00:57:13.143538 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:57:13.143514    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-27f57\" is forbidden: User \"system:node:ip-10-0-138-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-204.ec2.internal' and this object" podUID="d98235fe-14bd-497c-be43-6f124b60ccdd" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-27f57"
Sep 10 00:57:14.080129 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:57:14.080088    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-27f57\" is forbidden: User \"system:node:ip-10-0-138-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-204.ec2.internal' and this object" podUID="d98235fe-14bd-497c-be43-6f124b60ccdd" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-27f57"
Sep 10 00:57:14.082327 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:14.082306    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d98235fe-14bd-497c-be43-6f124b60ccdd" path="/var/lib/kubelet/pods/d98235fe-14bd-497c-be43-6f124b60ccdd/volumes"
Sep 10 00:57:24.132883 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:57:24.132845    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rzw4v"
Sep 10 00:58:33.957970 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:33.957933    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5f4bcfd7d9-nwd99"]
Sep 10 00:58:33.961449 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:33.961428    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f4bcfd7d9-nwd99"
Sep 10 00:58:33.964342 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:33.964261    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 00:58:33.964342 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:33.964294    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 00:58:33.964546 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:33.964274    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-p9kwl\""
Sep 10 00:58:33.965346 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:33.965329    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 00:58:33.972164 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:33.972140    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f4bcfd7d9-nwd99"]
Sep 10 00:58:34.089633 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:34.089604    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrb9d\" (UniqueName: \"kubernetes.io/projected/72673676-cbf5-4ad7-96c1-aac5e6ed7c71-kube-api-access-rrb9d\") pod \"maas-controller-5f4bcfd7d9-nwd99\" (UID: \"72673676-cbf5-4ad7-96c1-aac5e6ed7c71\") " pod="opendatahub/maas-controller-5f4bcfd7d9-nwd99"
Sep 10 00:58:34.089794 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:34.089638    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/72673676-cbf5-4ad7-96c1-aac5e6ed7c71-webhook-cert\") pod \"maas-controller-5f4bcfd7d9-nwd99\" (UID: \"72673676-cbf5-4ad7-96c1-aac5e6ed7c71\") " pod="opendatahub/maas-controller-5f4bcfd7d9-nwd99"
Sep 10 00:58:34.089794 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:34.089727    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72673676-cbf5-4ad7-96c1-aac5e6ed7c71-metrics-certs\") pod \"maas-controller-5f4bcfd7d9-nwd99\" (UID: \"72673676-cbf5-4ad7-96c1-aac5e6ed7c71\") " pod="opendatahub/maas-controller-5f4bcfd7d9-nwd99"
Sep 10 00:58:34.126719 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:34.126684    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-87f8d868d-c77l7"]
Sep 10 00:58:34.130531 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:34.130507    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-87f8d868d-c77l7"
Sep 10 00:58:34.138899 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:34.138876    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-87f8d868d-c77l7"]
Sep 10 00:58:34.190848 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:34.190638    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72673676-cbf5-4ad7-96c1-aac5e6ed7c71-metrics-certs\") pod \"maas-controller-5f4bcfd7d9-nwd99\" (UID: \"72673676-cbf5-4ad7-96c1-aac5e6ed7c71\") " pod="opendatahub/maas-controller-5f4bcfd7d9-nwd99"
Sep 10 00:58:34.190848 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:58:34.190754    2563 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 10 00:58:34.191139 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:58:34.190948    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72673676-cbf5-4ad7-96c1-aac5e6ed7c71-metrics-certs podName:72673676-cbf5-4ad7-96c1-aac5e6ed7c71 nodeName:}" failed. No retries permitted until 2026-09-10 00:58:34.690928054 +0000 UTC m=+531.188227616 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/72673676-cbf5-4ad7-96c1-aac5e6ed7c71-metrics-certs") pod "maas-controller-5f4bcfd7d9-nwd99" (UID: "72673676-cbf5-4ad7-96c1-aac5e6ed7c71") : secret "maas-controller-metrics-tls" not found
Sep 10 00:58:34.191283 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:34.191264    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrb9d\" (UniqueName: \"kubernetes.io/projected/72673676-cbf5-4ad7-96c1-aac5e6ed7c71-kube-api-access-rrb9d\") pod \"maas-controller-5f4bcfd7d9-nwd99\" (UID: \"72673676-cbf5-4ad7-96c1-aac5e6ed7c71\") " pod="opendatahub/maas-controller-5f4bcfd7d9-nwd99"
Sep 10 00:58:34.191348 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:34.191301    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/72673676-cbf5-4ad7-96c1-aac5e6ed7c71-webhook-cert\") pod \"maas-controller-5f4bcfd7d9-nwd99\" (UID: \"72673676-cbf5-4ad7-96c1-aac5e6ed7c71\") " pod="opendatahub/maas-controller-5f4bcfd7d9-nwd99"
Sep 10 00:58:34.193719 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:34.193696    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/72673676-cbf5-4ad7-96c1-aac5e6ed7c71-webhook-cert\") pod \"maas-controller-5f4bcfd7d9-nwd99\" (UID: \"72673676-cbf5-4ad7-96c1-aac5e6ed7c71\") " pod="opendatahub/maas-controller-5f4bcfd7d9-nwd99"
Sep 10 00:58:34.200388 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:34.200364    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrb9d\" (UniqueName: \"kubernetes.io/projected/72673676-cbf5-4ad7-96c1-aac5e6ed7c71-kube-api-access-rrb9d\") pod \"maas-controller-5f4bcfd7d9-nwd99\" (UID: \"72673676-cbf5-4ad7-96c1-aac5e6ed7c71\") " pod="opendatahub/maas-controller-5f4bcfd7d9-nwd99"
Sep 10 00:58:34.292794 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:34.292711    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2n4mh\" (UniqueName: \"kubernetes.io/projected/6c5f6fed-3b98-4569-8e26-e4918ff734a2-kube-api-access-2n4mh\") pod \"maas-controller-87f8d868d-c77l7\" (UID: \"6c5f6fed-3b98-4569-8e26-e4918ff734a2\") " pod="opendatahub/maas-controller-87f8d868d-c77l7"
Sep 10 00:58:34.292794 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:34.292749    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c5f6fed-3b98-4569-8e26-e4918ff734a2-webhook-cert\") pod \"maas-controller-87f8d868d-c77l7\" (UID: \"6c5f6fed-3b98-4569-8e26-e4918ff734a2\") " pod="opendatahub/maas-controller-87f8d868d-c77l7"
Sep 10 00:58:34.293011 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:34.292839    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c5f6fed-3b98-4569-8e26-e4918ff734a2-metrics-certs\") pod \"maas-controller-87f8d868d-c77l7\" (UID: \"6c5f6fed-3b98-4569-8e26-e4918ff734a2\") " pod="opendatahub/maas-controller-87f8d868d-c77l7"
Sep 10 00:58:34.394150 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:34.394109    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2n4mh\" (UniqueName: \"kubernetes.io/projected/6c5f6fed-3b98-4569-8e26-e4918ff734a2-kube-api-access-2n4mh\") pod \"maas-controller-87f8d868d-c77l7\" (UID: \"6c5f6fed-3b98-4569-8e26-e4918ff734a2\") " pod="opendatahub/maas-controller-87f8d868d-c77l7"
Sep 10 00:58:34.394150 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:34.394158    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c5f6fed-3b98-4569-8e26-e4918ff734a2-webhook-cert\") pod \"maas-controller-87f8d868d-c77l7\" (UID: \"6c5f6fed-3b98-4569-8e26-e4918ff734a2\") " pod="opendatahub/maas-controller-87f8d868d-c77l7"
Sep 10 00:58:34.394364 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:34.394190    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c5f6fed-3b98-4569-8e26-e4918ff734a2-metrics-certs\") pod \"maas-controller-87f8d868d-c77l7\" (UID: \"6c5f6fed-3b98-4569-8e26-e4918ff734a2\") " pod="opendatahub/maas-controller-87f8d868d-c77l7"
Sep 10 00:58:34.397076 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:34.397051    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c5f6fed-3b98-4569-8e26-e4918ff734a2-metrics-certs\") pod \"maas-controller-87f8d868d-c77l7\" (UID: \"6c5f6fed-3b98-4569-8e26-e4918ff734a2\") " pod="opendatahub/maas-controller-87f8d868d-c77l7"
Sep 10 00:58:34.397274 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:34.397251    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c5f6fed-3b98-4569-8e26-e4918ff734a2-webhook-cert\") pod \"maas-controller-87f8d868d-c77l7\" (UID: \"6c5f6fed-3b98-4569-8e26-e4918ff734a2\") " pod="opendatahub/maas-controller-87f8d868d-c77l7"
Sep 10 00:58:34.403005 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:34.402961    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2n4mh\" (UniqueName: \"kubernetes.io/projected/6c5f6fed-3b98-4569-8e26-e4918ff734a2-kube-api-access-2n4mh\") pod \"maas-controller-87f8d868d-c77l7\" (UID: \"6c5f6fed-3b98-4569-8e26-e4918ff734a2\") " pod="opendatahub/maas-controller-87f8d868d-c77l7"
Sep 10 00:58:34.442121 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:34.442089    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-87f8d868d-c77l7"
Sep 10 00:58:34.697578 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:34.697538    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72673676-cbf5-4ad7-96c1-aac5e6ed7c71-metrics-certs\") pod \"maas-controller-5f4bcfd7d9-nwd99\" (UID: \"72673676-cbf5-4ad7-96c1-aac5e6ed7c71\") " pod="opendatahub/maas-controller-5f4bcfd7d9-nwd99"
Sep 10 00:58:34.699950 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:34.699924    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72673676-cbf5-4ad7-96c1-aac5e6ed7c71-metrics-certs\") pod \"maas-controller-5f4bcfd7d9-nwd99\" (UID: \"72673676-cbf5-4ad7-96c1-aac5e6ed7c71\") " pod="opendatahub/maas-controller-5f4bcfd7d9-nwd99"
Sep 10 00:58:34.715738 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:34.715708    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-87f8d868d-c77l7"]
Sep 10 00:58:34.718880 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:58:34.718855    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c5f6fed_3b98_4569_8e26_e4918ff734a2.slice/crio-4912cfa5d71f6d282987fab6d782e9127c40812fc67ca4caeb131d309755f7c6 WatchSource:0}: Error finding container 4912cfa5d71f6d282987fab6d782e9127c40812fc67ca4caeb131d309755f7c6: Status 404 returned error can't find the container with id 4912cfa5d71f6d282987fab6d782e9127c40812fc67ca4caeb131d309755f7c6
Sep 10 00:58:34.873113 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:34.873072    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f4bcfd7d9-nwd99"
Sep 10 00:58:35.152345 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:35.152320    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f4bcfd7d9-nwd99"]
Sep 10 00:58:35.154928 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:58:35.154905    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72673676_cbf5_4ad7_96c1_aac5e6ed7c71.slice/crio-b795613459e68c2c4e0d680d9e6330ea065ea5e88a99204fe560577b1f7b39f1 WatchSource:0}: Error finding container b795613459e68c2c4e0d680d9e6330ea065ea5e88a99204fe560577b1f7b39f1: Status 404 returned error can't find the container with id b795613459e68c2c4e0d680d9e6330ea065ea5e88a99204fe560577b1f7b39f1
Sep 10 00:58:35.532741 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:35.532653    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f4bcfd7d9-nwd99" event={"ID":"72673676-cbf5-4ad7-96c1-aac5e6ed7c71","Type":"ContainerStarted","Data":"b795613459e68c2c4e0d680d9e6330ea065ea5e88a99204fe560577b1f7b39f1"}
Sep 10 00:58:35.534349 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:35.534271    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-87f8d868d-c77l7" event={"ID":"6c5f6fed-3b98-4569-8e26-e4918ff734a2","Type":"ContainerStarted","Data":"4912cfa5d71f6d282987fab6d782e9127c40812fc67ca4caeb131d309755f7c6"}
Sep 10 00:58:37.545869 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:37.545828    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f4bcfd7d9-nwd99" event={"ID":"72673676-cbf5-4ad7-96c1-aac5e6ed7c71","Type":"ContainerStarted","Data":"9693e59a668aee3e57a4052286e0d436c5901fcf9dcc6b7fe34f3f7ef75b7443"}
Sep 10 00:58:37.546337 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:37.545903    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5f4bcfd7d9-nwd99"
Sep 10 00:58:37.547897 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:37.547859    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-87f8d868d-c77l7" event={"ID":"6c5f6fed-3b98-4569-8e26-e4918ff734a2","Type":"ContainerStarted","Data":"6cf1344f7e970371d13d426a884a1d1c0cd243b140b2f8a094424b64b05ee1c0"}
Sep 10 00:58:37.548170 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:37.548138    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-87f8d868d-c77l7"
Sep 10 00:58:37.568431 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:37.568380    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5f4bcfd7d9-nwd99" podStartSLOduration=2.309041709 podStartE2EDuration="4.568363972s" totalImagesPullingTime="2.259322263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:58:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:58:37.567068235 +0000 UTC m=+534.064367817" watchObservedRunningTime="2026-09-10 00:58:37.568363972 +0000 UTC m=+534.065663550"
Sep 10 00:58:37.594592 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:37.594533    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-87f8d868d-c77l7" podStartSLOduration=0.905892476 podStartE2EDuration="3.59451103s" totalImagesPullingTime="2.688618554s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:58:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:58:37.592480578 +0000 UTC m=+534.089780163" watchObservedRunningTime="2026-09-10 00:58:37.59451103 +0000 UTC m=+534.091810610"
Sep 10 00:58:48.556208 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:48.556174    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5f4bcfd7d9-nwd99"
Sep 10 00:58:48.556584 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:48.556381    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-87f8d868d-c77l7"
Sep 10 00:58:48.615658 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:48.615620    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f4bcfd7d9-nwd99"]
Sep 10 00:58:48.616173 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:48.616110    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5f4bcfd7d9-nwd99" podUID="72673676-cbf5-4ad7-96c1-aac5e6ed7c71" containerName="manager" containerID="cri-o://9693e59a668aee3e57a4052286e0d436c5901fcf9dcc6b7fe34f3f7ef75b7443" gracePeriod=30
Sep 10 00:58:48.862231 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:48.862200    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f4bcfd7d9-nwd99"
Sep 10 00:58:49.032968 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:49.032934    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/72673676-cbf5-4ad7-96c1-aac5e6ed7c71-webhook-cert\") pod \"72673676-cbf5-4ad7-96c1-aac5e6ed7c71\" (UID: \"72673676-cbf5-4ad7-96c1-aac5e6ed7c71\") "
Sep 10 00:58:49.033177 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:49.033015    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rrb9d\" (UniqueName: \"kubernetes.io/projected/72673676-cbf5-4ad7-96c1-aac5e6ed7c71-kube-api-access-rrb9d\") pod \"72673676-cbf5-4ad7-96c1-aac5e6ed7c71\" (UID: \"72673676-cbf5-4ad7-96c1-aac5e6ed7c71\") "
Sep 10 00:58:49.033177 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:49.033080    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72673676-cbf5-4ad7-96c1-aac5e6ed7c71-metrics-certs\") pod \"72673676-cbf5-4ad7-96c1-aac5e6ed7c71\" (UID: \"72673676-cbf5-4ad7-96c1-aac5e6ed7c71\") "
Sep 10 00:58:49.035235 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:49.035203    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72673676-cbf5-4ad7-96c1-aac5e6ed7c71-kube-api-access-rrb9d" (OuterVolumeSpecName: "kube-api-access-rrb9d") pod "72673676-cbf5-4ad7-96c1-aac5e6ed7c71" (UID: "72673676-cbf5-4ad7-96c1-aac5e6ed7c71"). InnerVolumeSpecName "kube-api-access-rrb9d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 00:58:49.035359 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:49.035249    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72673676-cbf5-4ad7-96c1-aac5e6ed7c71-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "72673676-cbf5-4ad7-96c1-aac5e6ed7c71" (UID: "72673676-cbf5-4ad7-96c1-aac5e6ed7c71"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 00:58:49.035359 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:49.035329    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72673676-cbf5-4ad7-96c1-aac5e6ed7c71-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "72673676-cbf5-4ad7-96c1-aac5e6ed7c71" (UID: "72673676-cbf5-4ad7-96c1-aac5e6ed7c71"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 00:58:49.134496 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:49.134398    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72673676-cbf5-4ad7-96c1-aac5e6ed7c71-metrics-certs\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:58:49.134496 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:49.134432    2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/72673676-cbf5-4ad7-96c1-aac5e6ed7c71-webhook-cert\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:58:49.134496 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:49.134445    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrb9d\" (UniqueName: \"kubernetes.io/projected/72673676-cbf5-4ad7-96c1-aac5e6ed7c71-kube-api-access-rrb9d\") on node \"ip-10-0-138-204.ec2.internal\" DevicePath \"\""
Sep 10 00:58:49.607575 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:49.607541    2563 generic.go:356] "Generic (PLEG): container finished" podID="72673676-cbf5-4ad7-96c1-aac5e6ed7c71" containerID="9693e59a668aee3e57a4052286e0d436c5901fcf9dcc6b7fe34f3f7ef75b7443" exitCode=0
Sep 10 00:58:49.608001 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:49.607582    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f4bcfd7d9-nwd99"
Sep 10 00:58:49.608001 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:49.607628    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f4bcfd7d9-nwd99" event={"ID":"72673676-cbf5-4ad7-96c1-aac5e6ed7c71","Type":"ContainerDied","Data":"9693e59a668aee3e57a4052286e0d436c5901fcf9dcc6b7fe34f3f7ef75b7443"}
Sep 10 00:58:49.608001 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:49.607687    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f4bcfd7d9-nwd99" event={"ID":"72673676-cbf5-4ad7-96c1-aac5e6ed7c71","Type":"ContainerDied","Data":"b795613459e68c2c4e0d680d9e6330ea065ea5e88a99204fe560577b1f7b39f1"}
Sep 10 00:58:49.608001 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:49.607722    2563 scope.go:117] "RemoveContainer" containerID="9693e59a668aee3e57a4052286e0d436c5901fcf9dcc6b7fe34f3f7ef75b7443"
Sep 10 00:58:49.617385 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:49.617206    2563 scope.go:117] "RemoveContainer" containerID="9693e59a668aee3e57a4052286e0d436c5901fcf9dcc6b7fe34f3f7ef75b7443"
Sep 10 00:58:49.617513 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:58:49.617490    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9693e59a668aee3e57a4052286e0d436c5901fcf9dcc6b7fe34f3f7ef75b7443\": container with ID starting with 9693e59a668aee3e57a4052286e0d436c5901fcf9dcc6b7fe34f3f7ef75b7443 not found: ID does not exist" containerID="9693e59a668aee3e57a4052286e0d436c5901fcf9dcc6b7fe34f3f7ef75b7443"
Sep 10 00:58:49.617557 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:49.617523    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9693e59a668aee3e57a4052286e0d436c5901fcf9dcc6b7fe34f3f7ef75b7443"} err="failed to get container status \"9693e59a668aee3e57a4052286e0d436c5901fcf9dcc6b7fe34f3f7ef75b7443\": rpc error: code = NotFound desc = could not find container \"9693e59a668aee3e57a4052286e0d436c5901fcf9dcc6b7fe34f3f7ef75b7443\": container with ID starting with 9693e59a668aee3e57a4052286e0d436c5901fcf9dcc6b7fe34f3f7ef75b7443 not found: ID does not exist"
Sep 10 00:58:49.629843 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:49.629818    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f4bcfd7d9-nwd99"]
Sep 10 00:58:49.633524 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:49.633501    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5f4bcfd7d9-nwd99"]
Sep 10 00:58:50.082609 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:58:50.082530    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72673676-cbf5-4ad7-96c1-aac5e6ed7c71" path="/var/lib/kubelet/pods/72673676-cbf5-4ad7-96c1-aac5e6ed7c71/volumes"
Sep 10 00:59:10.689097 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:10.689064    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6dc78795f5-h2rxm"]
Sep 10 00:59:10.689588 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:10.689430    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72673676-cbf5-4ad7-96c1-aac5e6ed7c71" containerName="manager"
Sep 10 00:59:10.689588 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:10.689441    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="72673676-cbf5-4ad7-96c1-aac5e6ed7c71" containerName="manager"
Sep 10 00:59:10.689588 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:10.689505    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="72673676-cbf5-4ad7-96c1-aac5e6ed7c71" containerName="manager"
Sep 10 00:59:10.692625 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:10.692604    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6dc78795f5-h2rxm"
Sep 10 00:59:10.695294 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:10.695275    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 00:59:10.695401 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:10.695295    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 00:59:10.695495 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:10.695482    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 10 00:59:10.695545 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:10.695528    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 00:59:10.696737 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:10.696720    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nk2pd\""
Sep 10 00:59:10.700773 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:10.700735    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6dc78795f5-h2rxm"]
Sep 10 00:59:10.821062 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:10.821027    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2dc98840-2155-4e6c-b015-af85591fe40d-maas-api-tls\") pod \"maas-api-6dc78795f5-h2rxm\" (UID: \"2dc98840-2155-4e6c-b015-af85591fe40d\") " pod="odh-ai-gateway-infra/maas-api-6dc78795f5-h2rxm"
Sep 10 00:59:10.821062 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:10.821073    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dc98840-2155-4e6c-b015-af85591fe40d-openshift-service-ca\") pod \"maas-api-6dc78795f5-h2rxm\" (UID: \"2dc98840-2155-4e6c-b015-af85591fe40d\") " pod="odh-ai-gateway-infra/maas-api-6dc78795f5-h2rxm"
Sep 10 00:59:10.821273 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:10.821136    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2dc98840-2155-4e6c-b015-af85591fe40d-metrics-certs\") pod \"maas-api-6dc78795f5-h2rxm\" (UID: \"2dc98840-2155-4e6c-b015-af85591fe40d\") " pod="odh-ai-gateway-infra/maas-api-6dc78795f5-h2rxm"
Sep 10 00:59:10.821273 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:10.821226    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd59z\" (UniqueName: \"kubernetes.io/projected/2dc98840-2155-4e6c-b015-af85591fe40d-kube-api-access-nd59z\") pod \"maas-api-6dc78795f5-h2rxm\" (UID: \"2dc98840-2155-4e6c-b015-af85591fe40d\") " pod="odh-ai-gateway-infra/maas-api-6dc78795f5-h2rxm"
Sep 10 00:59:10.921869 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:10.921837    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nd59z\" (UniqueName: \"kubernetes.io/projected/2dc98840-2155-4e6c-b015-af85591fe40d-kube-api-access-nd59z\") pod \"maas-api-6dc78795f5-h2rxm\" (UID: \"2dc98840-2155-4e6c-b015-af85591fe40d\") " pod="odh-ai-gateway-infra/maas-api-6dc78795f5-h2rxm"
Sep 10 00:59:10.922060 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:10.921878    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2dc98840-2155-4e6c-b015-af85591fe40d-maas-api-tls\") pod \"maas-api-6dc78795f5-h2rxm\" (UID: \"2dc98840-2155-4e6c-b015-af85591fe40d\") " pod="odh-ai-gateway-infra/maas-api-6dc78795f5-h2rxm"
Sep 10 00:59:10.922060 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:10.921901    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dc98840-2155-4e6c-b015-af85591fe40d-openshift-service-ca\") pod \"maas-api-6dc78795f5-h2rxm\" (UID: \"2dc98840-2155-4e6c-b015-af85591fe40d\") " pod="odh-ai-gateway-infra/maas-api-6dc78795f5-h2rxm"
Sep 10 00:59:10.922060 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:10.921934    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2dc98840-2155-4e6c-b015-af85591fe40d-metrics-certs\") pod \"maas-api-6dc78795f5-h2rxm\" (UID: \"2dc98840-2155-4e6c-b015-af85591fe40d\") " pod="odh-ai-gateway-infra/maas-api-6dc78795f5-h2rxm"
Sep 10 00:59:10.922060 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:59:10.922018    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Sep 10 00:59:10.922280 ip-10-0-138-204 kubenswrapper[2563]: E0910 00:59:10.922099    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2dc98840-2155-4e6c-b015-af85591fe40d-maas-api-tls podName:2dc98840-2155-4e6c-b015-af85591fe40d nodeName:}" failed. No retries permitted until 2026-09-10 00:59:11.422079517 +0000 UTC m=+567.919379086 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2dc98840-2155-4e6c-b015-af85591fe40d-maas-api-tls") pod "maas-api-6dc78795f5-h2rxm" (UID: "2dc98840-2155-4e6c-b015-af85591fe40d") : secret "maas-api-serving-cert" not found
Sep 10 00:59:10.922729 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:10.922704    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dc98840-2155-4e6c-b015-af85591fe40d-openshift-service-ca\") pod \"maas-api-6dc78795f5-h2rxm\" (UID: \"2dc98840-2155-4e6c-b015-af85591fe40d\") " pod="odh-ai-gateway-infra/maas-api-6dc78795f5-h2rxm"
Sep 10 00:59:10.924504 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:10.924484    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2dc98840-2155-4e6c-b015-af85591fe40d-metrics-certs\") pod \"maas-api-6dc78795f5-h2rxm\" (UID: \"2dc98840-2155-4e6c-b015-af85591fe40d\") " pod="odh-ai-gateway-infra/maas-api-6dc78795f5-h2rxm"
Sep 10 00:59:10.930248 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:10.930224    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd59z\" (UniqueName: \"kubernetes.io/projected/2dc98840-2155-4e6c-b015-af85591fe40d-kube-api-access-nd59z\") pod \"maas-api-6dc78795f5-h2rxm\" (UID: \"2dc98840-2155-4e6c-b015-af85591fe40d\") " pod="odh-ai-gateway-infra/maas-api-6dc78795f5-h2rxm"
Sep 10 00:59:11.166150 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:11.166115    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-xzbz7"]
Sep 10 00:59:11.169888 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:11.169867    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-xzbz7"
Sep 10 00:59:11.173087 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:11.173060    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 10 00:59:11.173306 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:11.173270    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 10 00:59:11.173456 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:11.173151    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-xkrnh\""
Sep 10 00:59:11.173595 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:11.173578    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 10 00:59:11.180542 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:11.180518    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-xzbz7"]
Sep 10 00:59:11.224146 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:11.224109    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcac140e-784c-4cd5-ba9d-b7742fedc271-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-xzbz7\" (UID: \"dcac140e-784c-4cd5-ba9d-b7742fedc271\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-xzbz7"
Sep 10 00:59:11.224334 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:11.224151    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggg5f\" (UniqueName: \"kubernetes.io/projected/dcac140e-784c-4cd5-ba9d-b7742fedc271-kube-api-access-ggg5f\") pod \"payload-processing-5b54b7fbb7-xzbz7\" (UID: \"dcac140e-784c-4cd5-ba9d-b7742fedc271\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-xzbz7"
Sep 10 00:59:11.324937 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:11.324900    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcac140e-784c-4cd5-ba9d-b7742fedc271-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-xzbz7\" (UID: \"dcac140e-784c-4cd5-ba9d-b7742fedc271\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-xzbz7"
Sep 10 00:59:11.324937 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:11.324939    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggg5f\" (UniqueName: \"kubernetes.io/projected/dcac140e-784c-4cd5-ba9d-b7742fedc271-kube-api-access-ggg5f\") pod \"payload-processing-5b54b7fbb7-xzbz7\" (UID: \"dcac140e-784c-4cd5-ba9d-b7742fedc271\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-xzbz7"
Sep 10 00:59:11.325696 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:11.325673    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcac140e-784c-4cd5-ba9d-b7742fedc271-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-xzbz7\" (UID: \"dcac140e-784c-4cd5-ba9d-b7742fedc271\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-xzbz7"
Sep 10 00:59:11.337695 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:11.337664    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggg5f\" (UniqueName: \"kubernetes.io/projected/dcac140e-784c-4cd5-ba9d-b7742fedc271-kube-api-access-ggg5f\") pod \"payload-processing-5b54b7fbb7-xzbz7\" (UID: \"dcac140e-784c-4cd5-ba9d-b7742fedc271\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-xzbz7"
Sep 10 00:59:11.426270 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:11.426181    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2dc98840-2155-4e6c-b015-af85591fe40d-maas-api-tls\") pod \"maas-api-6dc78795f5-h2rxm\" (UID: \"2dc98840-2155-4e6c-b015-af85591fe40d\") " pod="odh-ai-gateway-infra/maas-api-6dc78795f5-h2rxm"
Sep 10 00:59:11.428565 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:11.428533    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2dc98840-2155-4e6c-b015-af85591fe40d-maas-api-tls\") pod \"maas-api-6dc78795f5-h2rxm\" (UID: \"2dc98840-2155-4e6c-b015-af85591fe40d\") " pod="odh-ai-gateway-infra/maas-api-6dc78795f5-h2rxm"
Sep 10 00:59:11.482289 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:11.482250    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-xzbz7"
Sep 10 00:59:11.604132 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:11.604095    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6dc78795f5-h2rxm"
Sep 10 00:59:11.752379 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:11.752307    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-xzbz7"]
Sep 10 00:59:11.754777 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:59:11.754749    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcac140e_784c_4cd5_ba9d_b7742fedc271.slice/crio-bde41e3fdfda424a0d3034c4e7b4e184b8a81a41adc0b67211f10ea0c52b122d WatchSource:0}: Error finding container bde41e3fdfda424a0d3034c4e7b4e184b8a81a41adc0b67211f10ea0c52b122d: Status 404 returned error can't find the container with id bde41e3fdfda424a0d3034c4e7b4e184b8a81a41adc0b67211f10ea0c52b122d
Sep 10 00:59:11.875787 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:11.875758    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6dc78795f5-h2rxm"]
Sep 10 00:59:11.878102 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:59:11.878075    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dc98840_2155_4e6c_b015_af85591fe40d.slice/crio-66cf3ce1a548fe7a993ebd46a7ce16a8cd222c073eeb20535476dbe60b68d1a6 WatchSource:0}: Error finding container 66cf3ce1a548fe7a993ebd46a7ce16a8cd222c073eeb20535476dbe60b68d1a6: Status 404 returned error can't find the container with id 66cf3ce1a548fe7a993ebd46a7ce16a8cd222c073eeb20535476dbe60b68d1a6
Sep 10 00:59:12.728496 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:12.728453    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-xzbz7" event={"ID":"dcac140e-784c-4cd5-ba9d-b7742fedc271","Type":"ContainerStarted","Data":"bde41e3fdfda424a0d3034c4e7b4e184b8a81a41adc0b67211f10ea0c52b122d"}
Sep 10 00:59:12.730444 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:12.730368    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6dc78795f5-h2rxm" event={"ID":"2dc98840-2155-4e6c-b015-af85591fe40d","Type":"ContainerStarted","Data":"66cf3ce1a548fe7a993ebd46a7ce16a8cd222c073eeb20535476dbe60b68d1a6"}
Sep 10 00:59:14.742545 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:14.742502    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-xzbz7" event={"ID":"dcac140e-784c-4cd5-ba9d-b7742fedc271","Type":"ContainerStarted","Data":"97cade006a85dc10102104a6071e2ed4cf37520a890f093f9f53a775445e55d5"}
Sep 10 00:59:14.743011 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:14.742721    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-xzbz7"
Sep 10 00:59:14.744130 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:14.744108    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6dc78795f5-h2rxm" event={"ID":"2dc98840-2155-4e6c-b015-af85591fe40d","Type":"ContainerStarted","Data":"82df62c2611df98a804f7a60e9b720576b477f671b8e2a67897a20f3eee1637b"}
Sep 10 00:59:14.744241 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:14.744230    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6dc78795f5-h2rxm"
Sep 10 00:59:14.758848 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:14.758805    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-xzbz7" podStartSLOduration=1.561388493 podStartE2EDuration="3.758791983s" totalImagesPullingTime="2.19740349s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:59:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:59:14.757831411 +0000 UTC m=+571.255130992" watchObservedRunningTime="2026-09-10 00:59:14.758791983 +0000 UTC m=+571.256091600"
Sep 10 00:59:14.775465 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:14.775416    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6dc78795f5-h2rxm" podStartSLOduration=2.700845684 podStartE2EDuration="4.775398078s" totalImagesPullingTime="2.074552394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:59:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:59:14.774757354 +0000 UTC m=+571.272056936" watchObservedRunningTime="2026-09-10 00:59:14.775398078 +0000 UTC m=+571.272697657"
Sep 10 00:59:20.754052 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:20.754022    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6dc78795f5-h2rxm"
Sep 10 00:59:25.750874 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:25.750841    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-xzbz7"
Sep 10 00:59:43.487048 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:43.487004    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr"]
Sep 10 00:59:43.491323 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:43.491298    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr"
Sep 10 00:59:43.494898 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:43.494871    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 10 00:59:43.495065 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:43.494909    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 10 00:59:43.495065 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:43.494960    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 10 00:59:43.495300 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:43.495279    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-sbt7f\""
Sep 10 00:59:43.498662 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:43.498640    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr"]
Sep 10 00:59:43.511491 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:43.511463    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ml6lp\" (UniqueName: \"kubernetes.io/projected/cd2a4e9a-f207-4b1d-998a-591801f95098-kube-api-access-ml6lp\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr\" (UID: \"cd2a4e9a-f207-4b1d-998a-591801f95098\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr"
Sep 10 00:59:43.511642 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:43.511512    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd2a4e9a-f207-4b1d-998a-591801f95098-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr\" (UID: \"cd2a4e9a-f207-4b1d-998a-591801f95098\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr"
Sep 10 00:59:43.511642 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:43.511532    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd2a4e9a-f207-4b1d-998a-591801f95098-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr\" (UID: \"cd2a4e9a-f207-4b1d-998a-591801f95098\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr"
Sep 10 00:59:43.511642 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:43.511604    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd2a4e9a-f207-4b1d-998a-591801f95098-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr\" (UID: \"cd2a4e9a-f207-4b1d-998a-591801f95098\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr"
Sep 10 00:59:43.511801 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:43.511643    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd2a4e9a-f207-4b1d-998a-591801f95098-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr\" (UID: \"cd2a4e9a-f207-4b1d-998a-591801f95098\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr"
Sep 10 00:59:43.511801 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:43.511741    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd2a4e9a-f207-4b1d-998a-591801f95098-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr\" (UID: \"cd2a4e9a-f207-4b1d-998a-591801f95098\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr"
Sep 10 00:59:43.612688 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:43.612648    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd2a4e9a-f207-4b1d-998a-591801f95098-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr\" (UID: \"cd2a4e9a-f207-4b1d-998a-591801f95098\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr"
Sep 10 00:59:43.612880 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:43.612704    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd2a4e9a-f207-4b1d-998a-591801f95098-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr\" (UID: \"cd2a4e9a-f207-4b1d-998a-591801f95098\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr"
Sep 10 00:59:43.612880 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:43.612761    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd2a4e9a-f207-4b1d-998a-591801f95098-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr\" (UID: \"cd2a4e9a-f207-4b1d-998a-591801f95098\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr"
Sep 10 00:59:43.612880 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:43.612790    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ml6lp\" (UniqueName: \"kubernetes.io/projected/cd2a4e9a-f207-4b1d-998a-591801f95098-kube-api-access-ml6lp\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr\" (UID: \"cd2a4e9a-f207-4b1d-998a-591801f95098\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr"
Sep 10 00:59:43.612880 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:43.612834    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd2a4e9a-f207-4b1d-998a-591801f95098-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr\" (UID: \"cd2a4e9a-f207-4b1d-998a-591801f95098\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr"
Sep 10 00:59:43.612880 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:43.612856    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd2a4e9a-f207-4b1d-998a-591801f95098-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr\" (UID: \"cd2a4e9a-f207-4b1d-998a-591801f95098\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr"
Sep 10 00:59:43.613527 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:43.613109    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd2a4e9a-f207-4b1d-998a-591801f95098-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr\" (UID: \"cd2a4e9a-f207-4b1d-998a-591801f95098\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr"
Sep 10 00:59:43.613527 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:43.613160    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd2a4e9a-f207-4b1d-998a-591801f95098-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr\" (UID: \"cd2a4e9a-f207-4b1d-998a-591801f95098\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr"
Sep 10 00:59:43.613527 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:43.613206    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd2a4e9a-f207-4b1d-998a-591801f95098-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr\" (UID: \"cd2a4e9a-f207-4b1d-998a-591801f95098\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr"
Sep 10 00:59:43.615092 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:43.615065    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd2a4e9a-f207-4b1d-998a-591801f95098-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr\" (UID: \"cd2a4e9a-f207-4b1d-998a-591801f95098\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr"
Sep 10 00:59:43.615325 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:43.615307    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd2a4e9a-f207-4b1d-998a-591801f95098-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr\" (UID: \"cd2a4e9a-f207-4b1d-998a-591801f95098\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr"
Sep 10 00:59:43.620817 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:43.620797    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ml6lp\" (UniqueName: \"kubernetes.io/projected/cd2a4e9a-f207-4b1d-998a-591801f95098-kube-api-access-ml6lp\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr\" (UID: \"cd2a4e9a-f207-4b1d-998a-591801f95098\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr"
Sep 10 00:59:43.805119 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:43.805029    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr"
Sep 10 00:59:43.978539 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:43.978504    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nnwxp_1cee7771-6381-4fad-8c1e-ad1098fe409b/console-operator/1.log"
Sep 10 00:59:43.978694 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:43.978515    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nnwxp_1cee7771-6381-4fad-8c1e-ad1098fe409b/console-operator/1.log"
Sep 10 00:59:44.076366 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:44.076288    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr"]
Sep 10 00:59:44.080872 ip-10-0-138-204 kubenswrapper[2563]: W0910 00:59:44.080843    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd2a4e9a_f207_4b1d_998a_591801f95098.slice/crio-e8ef05a17a6252bb8027b26a3c57ad891cc4c3eccf1b2f2d94d3c0df7e5b36d2 WatchSource:0}: Error finding container e8ef05a17a6252bb8027b26a3c57ad891cc4c3eccf1b2f2d94d3c0df7e5b36d2: Status 404 returned error can't find the container with id e8ef05a17a6252bb8027b26a3c57ad891cc4c3eccf1b2f2d94d3c0df7e5b36d2
Sep 10 00:59:44.898359 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:44.898320    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr" event={"ID":"cd2a4e9a-f207-4b1d-998a-591801f95098","Type":"ContainerStarted","Data":"e8ef05a17a6252bb8027b26a3c57ad891cc4c3eccf1b2f2d94d3c0df7e5b36d2"}
Sep 10 00:59:45.905711 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:45.905626    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr" event={"ID":"cd2a4e9a-f207-4b1d-998a-591801f95098","Type":"ContainerStarted","Data":"3cb37fe4a716a84c15673de43b46b3badcb24492b05906baeec6568cb3984118"}
Sep 10 00:59:45.922816 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:45.922767    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr" podStartSLOduration=1.407809259 podStartE2EDuration="2.922748838s" totalImagesPullingTime="1.514939579s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 00:59:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 00:59:45.921757084 +0000 UTC m=+602.419056664" watchObservedRunningTime="2026-09-10 00:59:45.922748838 +0000 UTC m=+602.420048416"
Sep 10 00:59:53.805950 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:53.805900    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr"
Sep 10 00:59:53.806450 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:53.805980    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr"
Sep 10 00:59:53.818547 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:53.818518    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr"
Sep 10 00:59:53.956128 ip-10-0-138-204 kubenswrapper[2563]: I0910 00:59:53.956097    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr"
Sep 10 01:00:11.387212 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:11.387131    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"]
Sep 10 01:00:11.390609 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:11.390589    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:00:11.393493 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:11.393467    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 10 01:00:11.403443 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:11.403416    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"]
Sep 10 01:00:11.487608 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:11.487571    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d66ca905-636b-4e4e-94ce-e26913e3f8a6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t\" (UID: \"d66ca905-636b-4e4e-94ce-e26913e3f8a6\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:00:11.487608 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:11.487610    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d66ca905-636b-4e4e-94ce-e26913e3f8a6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t\" (UID: \"d66ca905-636b-4e4e-94ce-e26913e3f8a6\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:00:11.487813 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:11.487634    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d66ca905-636b-4e4e-94ce-e26913e3f8a6-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t\" (UID: \"d66ca905-636b-4e4e-94ce-e26913e3f8a6\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:00:11.487813 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:11.487710    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d66ca905-636b-4e4e-94ce-e26913e3f8a6-dshm\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t\" (UID: \"d66ca905-636b-4e4e-94ce-e26913e3f8a6\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:00:11.487813 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:11.487754    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d66ca905-636b-4e4e-94ce-e26913e3f8a6-home\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t\" (UID: \"d66ca905-636b-4e4e-94ce-e26913e3f8a6\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:00:11.487813 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:11.487771    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hg867\" (UniqueName: \"kubernetes.io/projected/d66ca905-636b-4e4e-94ce-e26913e3f8a6-kube-api-access-hg867\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t\" (UID: \"d66ca905-636b-4e4e-94ce-e26913e3f8a6\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:00:11.589220 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:11.589160    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d66ca905-636b-4e4e-94ce-e26913e3f8a6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t\" (UID: \"d66ca905-636b-4e4e-94ce-e26913e3f8a6\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:00:11.589220 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:11.589218    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d66ca905-636b-4e4e-94ce-e26913e3f8a6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t\" (UID: \"d66ca905-636b-4e4e-94ce-e26913e3f8a6\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:00:11.589481 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:11.589248    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d66ca905-636b-4e4e-94ce-e26913e3f8a6-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t\" (UID: \"d66ca905-636b-4e4e-94ce-e26913e3f8a6\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:00:11.589481 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:11.589306    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d66ca905-636b-4e4e-94ce-e26913e3f8a6-dshm\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t\" (UID: \"d66ca905-636b-4e4e-94ce-e26913e3f8a6\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:00:11.589481 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:11.589360    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d66ca905-636b-4e4e-94ce-e26913e3f8a6-home\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t\" (UID: \"d66ca905-636b-4e4e-94ce-e26913e3f8a6\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:00:11.589481 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:11.589386    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg867\" (UniqueName: \"kubernetes.io/projected/d66ca905-636b-4e4e-94ce-e26913e3f8a6-kube-api-access-hg867\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t\" (UID: \"d66ca905-636b-4e4e-94ce-e26913e3f8a6\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:00:11.589693 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:11.589583    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d66ca905-636b-4e4e-94ce-e26913e3f8a6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t\" (UID: \"d66ca905-636b-4e4e-94ce-e26913e3f8a6\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:00:11.589693 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:11.589645    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d66ca905-636b-4e4e-94ce-e26913e3f8a6-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t\" (UID: \"d66ca905-636b-4e4e-94ce-e26913e3f8a6\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:00:11.589693 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:11.589681    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d66ca905-636b-4e4e-94ce-e26913e3f8a6-home\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t\" (UID: \"d66ca905-636b-4e4e-94ce-e26913e3f8a6\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:00:11.591697 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:11.591671    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d66ca905-636b-4e4e-94ce-e26913e3f8a6-dshm\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t\" (UID: \"d66ca905-636b-4e4e-94ce-e26913e3f8a6\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:00:11.592016 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:11.591976    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d66ca905-636b-4e4e-94ce-e26913e3f8a6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t\" (UID: \"d66ca905-636b-4e4e-94ce-e26913e3f8a6\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:00:11.597762 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:11.597740    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hg867\" (UniqueName: \"kubernetes.io/projected/d66ca905-636b-4e4e-94ce-e26913e3f8a6-kube-api-access-hg867\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t\" (UID: \"d66ca905-636b-4e4e-94ce-e26913e3f8a6\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:00:11.703555 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:11.703473    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:00:11.975649 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:11.975618    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"]
Sep 10 01:00:11.977432 ip-10-0-138-204 kubenswrapper[2563]: W0910 01:00:11.977403    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd66ca905_636b_4e4e_94ce_e26913e3f8a6.slice/crio-d898796a55fe2b02458b89100ee218adcc2922d7c62010480f88f09c8b7e5642 WatchSource:0}: Error finding container d898796a55fe2b02458b89100ee218adcc2922d7c62010480f88f09c8b7e5642: Status 404 returned error can't find the container with id d898796a55fe2b02458b89100ee218adcc2922d7c62010480f88f09c8b7e5642
Sep 10 01:00:11.978767 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:11.978748    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 01:00:12.041569 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:12.041535    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t" event={"ID":"d66ca905-636b-4e4e-94ce-e26913e3f8a6","Type":"ContainerStarted","Data":"d898796a55fe2b02458b89100ee218adcc2922d7c62010480f88f09c8b7e5642"}
Sep 10 01:00:14.053462 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:14.053407    2563 generic.go:356] "Generic (PLEG): container finished" podID="d66ca905-636b-4e4e-94ce-e26913e3f8a6" containerID="5aebc43a9540c6e055b15e0139b27f6c09eccf63ea55c2dee9e7afdca25fcded" exitCode=0
Sep 10 01:00:14.053923 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:14.053499    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t" event={"ID":"d66ca905-636b-4e4e-94ce-e26913e3f8a6","Type":"ContainerDied","Data":"5aebc43a9540c6e055b15e0139b27f6c09eccf63ea55c2dee9e7afdca25fcded"}
Sep 10 01:00:14.053923 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:14.053761    2563 scope.go:117] "RemoveContainer" containerID="5aebc43a9540c6e055b15e0139b27f6c09eccf63ea55c2dee9e7afdca25fcded"
Sep 10 01:00:15.060620 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:15.060586    2563 generic.go:356] "Generic (PLEG): container finished" podID="d66ca905-636b-4e4e-94ce-e26913e3f8a6" containerID="af03707db36e5d4a9359aa1ec612ef5a2d6c55ca6f69ca8d3d09ceff3de0122a" exitCode=0
Sep 10 01:00:15.061027 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:15.060676    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t" event={"ID":"d66ca905-636b-4e4e-94ce-e26913e3f8a6","Type":"ContainerDied","Data":"af03707db36e5d4a9359aa1ec612ef5a2d6c55ca6f69ca8d3d09ceff3de0122a"}
Sep 10 01:00:15.061027 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:15.060735    2563 scope.go:117] "RemoveContainer" containerID="5aebc43a9540c6e055b15e0139b27f6c09eccf63ea55c2dee9e7afdca25fcded"
Sep 10 01:00:15.061027 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:15.060930    2563 scope.go:117] "RemoveContainer" containerID="af03707db36e5d4a9359aa1ec612ef5a2d6c55ca6f69ca8d3d09ceff3de0122a"
Sep 10 01:00:15.061171 ip-10-0-138-204 kubenswrapper[2563]: E0910 01:00:15.061153    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t_llm(d66ca905-636b-4e4e-94ce-e26913e3f8a6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t" podUID="d66ca905-636b-4e4e-94ce-e26913e3f8a6"
Sep 10 01:00:16.067488 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:16.067458    2563 scope.go:117] "RemoveContainer" containerID="af03707db36e5d4a9359aa1ec612ef5a2d6c55ca6f69ca8d3d09ceff3de0122a"
Sep 10 01:00:16.067861 ip-10-0-138-204 kubenswrapper[2563]: E0910 01:00:16.067634    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t_llm(d66ca905-636b-4e4e-94ce-e26913e3f8a6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t" podUID="d66ca905-636b-4e4e-94ce-e26913e3f8a6"
Sep 10 01:00:21.704552 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:21.704508    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:00:21.704949 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:21.704579    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:00:21.704949 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:21.704596    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:00:21.704949 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:21.704608    2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:00:21.705105 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:21.705016    2563 scope.go:117] "RemoveContainer" containerID="af03707db36e5d4a9359aa1ec612ef5a2d6c55ca6f69ca8d3d09ceff3de0122a"
Sep 10 01:00:21.705214 ip-10-0-138-204 kubenswrapper[2563]: E0910 01:00:21.705195    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t_llm(d66ca905-636b-4e4e-94ce-e26913e3f8a6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t" podUID="d66ca905-636b-4e4e-94ce-e26913e3f8a6"
Sep 10 01:00:34.079512 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:34.079480    2563 scope.go:117] "RemoveContainer" containerID="af03707db36e5d4a9359aa1ec612ef5a2d6c55ca6f69ca8d3d09ceff3de0122a"
Sep 10 01:00:35.179687 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:35.179657    2563 generic.go:356] "Generic (PLEG): container finished" podID="d66ca905-636b-4e4e-94ce-e26913e3f8a6" containerID="fd2f6520b7e37c362115e87706d0c0f6ea6ebdc928cba5045776c87d01ea4f5d" exitCode=0
Sep 10 01:00:35.180177 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:35.179734    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t" event={"ID":"d66ca905-636b-4e4e-94ce-e26913e3f8a6","Type":"ContainerDied","Data":"fd2f6520b7e37c362115e87706d0c0f6ea6ebdc928cba5045776c87d01ea4f5d"}
Sep 10 01:00:35.180177 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:35.179813    2563 scope.go:117] "RemoveContainer" containerID="af03707db36e5d4a9359aa1ec612ef5a2d6c55ca6f69ca8d3d09ceff3de0122a"
Sep 10 01:00:35.180277 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:35.180241    2563 scope.go:117] "RemoveContainer" containerID="fd2f6520b7e37c362115e87706d0c0f6ea6ebdc928cba5045776c87d01ea4f5d"
Sep 10 01:00:35.180467 ip-10-0-138-204 kubenswrapper[2563]: E0910 01:00:35.180449    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t_llm(d66ca905-636b-4e4e-94ce-e26913e3f8a6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t" podUID="d66ca905-636b-4e4e-94ce-e26913e3f8a6"
Sep 10 01:00:41.704197 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:41.704157    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:00:41.704576 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:41.704226    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:00:41.704576 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:41.704244    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:00:41.704576 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:41.704257    2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:00:41.704703 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:41.704686    2563 scope.go:117] "RemoveContainer" containerID="fd2f6520b7e37c362115e87706d0c0f6ea6ebdc928cba5045776c87d01ea4f5d"
Sep 10 01:00:41.704913 ip-10-0-138-204 kubenswrapper[2563]: E0910 01:00:41.704895    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t_llm(d66ca905-636b-4e4e-94ce-e26913e3f8a6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t" podUID="d66ca905-636b-4e4e-94ce-e26913e3f8a6"
Sep 10 01:00:53.083429 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:00:53.083396    2563 scope.go:117] "RemoveContainer" containerID="fd2f6520b7e37c362115e87706d0c0f6ea6ebdc928cba5045776c87d01ea4f5d"
Sep 10 01:00:53.083912 ip-10-0-138-204 kubenswrapper[2563]: E0910 01:00:53.083567    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t_llm(d66ca905-636b-4e4e-94ce-e26913e3f8a6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t" podUID="d66ca905-636b-4e4e-94ce-e26913e3f8a6"
Sep 10 01:01:06.076949 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:01:06.076906    2563 scope.go:117] "RemoveContainer" containerID="fd2f6520b7e37c362115e87706d0c0f6ea6ebdc928cba5045776c87d01ea4f5d"
Sep 10 01:01:06.339055 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:01:06.339021    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t" event={"ID":"d66ca905-636b-4e4e-94ce-e26913e3f8a6","Type":"ContainerStarted","Data":"1b426eb798c795cb988d24f4e1e0a6eb7a4aca2814dd0d2ae1e3e54ce4764f34"}
Sep 10 01:01:06.359658 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:01:06.359609    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t" podStartSLOduration=53.648875113 podStartE2EDuration="55.359591405s" totalImagesPullingTime="1.710716292s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 01:00:11 +0000 UTC" imagePullSessionsCount=4 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 01:01:06.357675148 +0000 UTC m=+682.854974752" watchObservedRunningTime="2026-09-10 01:01:06.359591405 +0000 UTC m=+682.856891014"
Sep 10 01:01:07.347605 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:01:07.347572    2563 generic.go:356] "Generic (PLEG): container finished" podID="d66ca905-636b-4e4e-94ce-e26913e3f8a6" containerID="1b426eb798c795cb988d24f4e1e0a6eb7a4aca2814dd0d2ae1e3e54ce4764f34" exitCode=0
Sep 10 01:01:07.348040 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:01:07.347614    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t" event={"ID":"d66ca905-636b-4e4e-94ce-e26913e3f8a6","Type":"ContainerDied","Data":"1b426eb798c795cb988d24f4e1e0a6eb7a4aca2814dd0d2ae1e3e54ce4764f34"}
Sep 10 01:01:07.348040 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:01:07.347694    2563 scope.go:117] "RemoveContainer" containerID="fd2f6520b7e37c362115e87706d0c0f6ea6ebdc928cba5045776c87d01ea4f5d"
Sep 10 01:01:07.348162 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:01:07.348079    2563 scope.go:117] "RemoveContainer" containerID="1b426eb798c795cb988d24f4e1e0a6eb7a4aca2814dd0d2ae1e3e54ce4764f34"
Sep 10 01:01:07.348301 ip-10-0-138-204 kubenswrapper[2563]: E0910 01:01:07.348274    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t_llm(d66ca905-636b-4e4e-94ce-e26913e3f8a6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t" podUID="d66ca905-636b-4e4e-94ce-e26913e3f8a6"
Sep 10 01:01:11.703972 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:01:11.703938    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:01:11.704383 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:01:11.704013    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:01:11.704383 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:01:11.704032    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:01:11.704383 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:01:11.704046    2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:01:11.704502 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:01:11.704406    2563 scope.go:117] "RemoveContainer" containerID="1b426eb798c795cb988d24f4e1e0a6eb7a4aca2814dd0d2ae1e3e54ce4764f34"
Sep 10 01:01:11.704613 ip-10-0-138-204 kubenswrapper[2563]: E0910 01:01:11.704593    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t_llm(d66ca905-636b-4e4e-94ce-e26913e3f8a6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t" podUID="d66ca905-636b-4e4e-94ce-e26913e3f8a6"
Sep 10 01:01:25.077047 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:01:25.077009    2563 scope.go:117] "RemoveContainer" containerID="1b426eb798c795cb988d24f4e1e0a6eb7a4aca2814dd0d2ae1e3e54ce4764f34"
Sep 10 01:01:25.077483 ip-10-0-138-204 kubenswrapper[2563]: E0910 01:01:25.077224    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t_llm(d66ca905-636b-4e4e-94ce-e26913e3f8a6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t" podUID="d66ca905-636b-4e4e-94ce-e26913e3f8a6"
Sep 10 01:01:39.077332 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:01:39.077240    2563 scope.go:117] "RemoveContainer" containerID="1b426eb798c795cb988d24f4e1e0a6eb7a4aca2814dd0d2ae1e3e54ce4764f34"
Sep 10 01:01:39.077812 ip-10-0-138-204 kubenswrapper[2563]: E0910 01:01:39.077424    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t_llm(d66ca905-636b-4e4e-94ce-e26913e3f8a6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t" podUID="d66ca905-636b-4e4e-94ce-e26913e3f8a6"
Sep 10 01:01:50.076907 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:01:50.076865    2563 scope.go:117] "RemoveContainer" containerID="1b426eb798c795cb988d24f4e1e0a6eb7a4aca2814dd0d2ae1e3e54ce4764f34"
Sep 10 01:01:50.566009 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:01:50.565950    2563 generic.go:356] "Generic (PLEG): container finished" podID="d66ca905-636b-4e4e-94ce-e26913e3f8a6" containerID="a3cdfa86d2f270f52b20332084e1cfaa7f37b13e897f4d0354d404f479fa3dcb" exitCode=0
Sep 10 01:01:50.566203 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:01:50.566023    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t" event={"ID":"d66ca905-636b-4e4e-94ce-e26913e3f8a6","Type":"ContainerDied","Data":"a3cdfa86d2f270f52b20332084e1cfaa7f37b13e897f4d0354d404f479fa3dcb"}
Sep 10 01:01:50.566203 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:01:50.566085    2563 scope.go:117] "RemoveContainer" containerID="1b426eb798c795cb988d24f4e1e0a6eb7a4aca2814dd0d2ae1e3e54ce4764f34"
Sep 10 01:01:50.566464 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:01:50.566449    2563 scope.go:117] "RemoveContainer" containerID="a3cdfa86d2f270f52b20332084e1cfaa7f37b13e897f4d0354d404f479fa3dcb"
Sep 10 01:01:50.566715 ip-10-0-138-204 kubenswrapper[2563]: E0910 01:01:50.566696    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t_llm(d66ca905-636b-4e4e-94ce-e26913e3f8a6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t" podUID="d66ca905-636b-4e4e-94ce-e26913e3f8a6"
Sep 10 01:01:51.704512 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:01:51.704477    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:01:51.704950 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:01:51.704536    2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:01:51.704950 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:01:51.704554    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:01:51.704950 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:01:51.704569    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:01:51.704950 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:01:51.704931    2563 scope.go:117] "RemoveContainer" containerID="a3cdfa86d2f270f52b20332084e1cfaa7f37b13e897f4d0354d404f479fa3dcb"
Sep 10 01:01:51.705173 ip-10-0-138-204 kubenswrapper[2563]: E0910 01:01:51.705152    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t_llm(d66ca905-636b-4e4e-94ce-e26913e3f8a6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t" podUID="d66ca905-636b-4e4e-94ce-e26913e3f8a6"
Sep 10 01:02:06.076426 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:02:06.076397    2563 scope.go:117] "RemoveContainer" containerID="a3cdfa86d2f270f52b20332084e1cfaa7f37b13e897f4d0354d404f479fa3dcb"
Sep 10 01:02:06.076881 ip-10-0-138-204 kubenswrapper[2563]: E0910 01:02:06.076583    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t_llm(d66ca905-636b-4e4e-94ce-e26913e3f8a6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t" podUID="d66ca905-636b-4e4e-94ce-e26913e3f8a6"
Sep 10 01:02:21.077254 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:02:21.077223    2563 scope.go:117] "RemoveContainer" containerID="a3cdfa86d2f270f52b20332084e1cfaa7f37b13e897f4d0354d404f479fa3dcb"
Sep 10 01:02:21.077716 ip-10-0-138-204 kubenswrapper[2563]: E0910 01:02:21.077408    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t_llm(d66ca905-636b-4e4e-94ce-e26913e3f8a6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t" podUID="d66ca905-636b-4e4e-94ce-e26913e3f8a6"
Sep 10 01:02:36.076758 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:02:36.076725    2563 scope.go:117] "RemoveContainer" containerID="a3cdfa86d2f270f52b20332084e1cfaa7f37b13e897f4d0354d404f479fa3dcb"
Sep 10 01:02:36.077231 ip-10-0-138-204 kubenswrapper[2563]: E0910 01:02:36.076903    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t_llm(d66ca905-636b-4e4e-94ce-e26913e3f8a6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t" podUID="d66ca905-636b-4e4e-94ce-e26913e3f8a6"
Sep 10 01:02:51.076829 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:02:51.076791    2563 scope.go:117] "RemoveContainer" containerID="a3cdfa86d2f270f52b20332084e1cfaa7f37b13e897f4d0354d404f479fa3dcb"
Sep 10 01:02:51.077311 ip-10-0-138-204 kubenswrapper[2563]: E0910 01:02:51.076979    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t_llm(d66ca905-636b-4e4e-94ce-e26913e3f8a6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t" podUID="d66ca905-636b-4e4e-94ce-e26913e3f8a6"
Sep 10 01:03:03.076494 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:03:03.076461    2563 scope.go:117] "RemoveContainer" containerID="a3cdfa86d2f270f52b20332084e1cfaa7f37b13e897f4d0354d404f479fa3dcb"
Sep 10 01:03:03.076944 ip-10-0-138-204 kubenswrapper[2563]: E0910 01:03:03.076658    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t_llm(d66ca905-636b-4e4e-94ce-e26913e3f8a6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t" podUID="d66ca905-636b-4e4e-94ce-e26913e3f8a6"
Sep 10 01:03:17.077538 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:03:17.077442    2563 scope.go:117] "RemoveContainer" containerID="a3cdfa86d2f270f52b20332084e1cfaa7f37b13e897f4d0354d404f479fa3dcb"
Sep 10 01:03:18.013684 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:03:18.013649    2563 generic.go:356] "Generic (PLEG): container finished" podID="d66ca905-636b-4e4e-94ce-e26913e3f8a6" containerID="b090d8d22885556b1950653a061fee464158d77029123f4387435a1bac0a87ed" exitCode=0
Sep 10 01:03:18.013850 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:03:18.013723    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t" event={"ID":"d66ca905-636b-4e4e-94ce-e26913e3f8a6","Type":"ContainerDied","Data":"b090d8d22885556b1950653a061fee464158d77029123f4387435a1bac0a87ed"}
Sep 10 01:03:18.013850 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:03:18.013815    2563 scope.go:117] "RemoveContainer" containerID="a3cdfa86d2f270f52b20332084e1cfaa7f37b13e897f4d0354d404f479fa3dcb"
Sep 10 01:03:18.014182 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:03:18.014162    2563 scope.go:117] "RemoveContainer" containerID="b090d8d22885556b1950653a061fee464158d77029123f4387435a1bac0a87ed"
Sep 10 01:03:18.014414 ip-10-0-138-204 kubenswrapper[2563]: E0910 01:03:18.014391    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t_llm(d66ca905-636b-4e4e-94ce-e26913e3f8a6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t" podUID="d66ca905-636b-4e4e-94ce-e26913e3f8a6"
Sep 10 01:03:21.704200 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:03:21.704159    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:03:21.704611 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:03:21.704226    2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:03:21.704611 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:03:21.704244    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:03:21.704611 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:03:21.704259    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t"
Sep 10 01:03:21.704611 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:03:21.704588    2563 scope.go:117] "RemoveContainer" containerID="b090d8d22885556b1950653a061fee464158d77029123f4387435a1bac0a87ed"
Sep 10 01:03:21.704831 ip-10-0-138-204 kubenswrapper[2563]: E0910 01:03:21.704807    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t_llm(d66ca905-636b-4e4e-94ce-e26913e3f8a6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t" podUID="d66ca905-636b-4e4e-94ce-e26913e3f8a6"
Sep 10 01:03:36.083488 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:03:36.083455    2563 scope.go:117] "RemoveContainer" containerID="b090d8d22885556b1950653a061fee464158d77029123f4387435a1bac0a87ed"
Sep 10 01:03:36.083940 ip-10-0-138-204 kubenswrapper[2563]: E0910 01:03:36.083669    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t_llm(d66ca905-636b-4e4e-94ce-e26913e3f8a6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t" podUID="d66ca905-636b-4e4e-94ce-e26913e3f8a6"
Sep 10 01:03:50.076844 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:03:50.076811    2563 scope.go:117] "RemoveContainer" containerID="b090d8d22885556b1950653a061fee464158d77029123f4387435a1bac0a87ed"
Sep 10 01:03:50.077334 ip-10-0-138-204 kubenswrapper[2563]: E0910 01:03:50.077054    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t_llm(d66ca905-636b-4e4e-94ce-e26913e3f8a6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t" podUID="d66ca905-636b-4e4e-94ce-e26913e3f8a6"
Sep 10 01:04:05.077104 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:05.077067    2563 scope.go:117] "RemoveContainer" containerID="b090d8d22885556b1950653a061fee464158d77029123f4387435a1bac0a87ed"
Sep 10 01:04:05.077584 ip-10-0-138-204 kubenswrapper[2563]: E0910 01:04:05.077265    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t_llm(d66ca905-636b-4e4e-94ce-e26913e3f8a6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t" podUID="d66ca905-636b-4e4e-94ce-e26913e3f8a6"
Sep 10 01:04:16.077276 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:16.077243    2563 scope.go:117] "RemoveContainer" containerID="b090d8d22885556b1950653a061fee464158d77029123f4387435a1bac0a87ed"
Sep 10 01:04:16.077739 ip-10-0-138-204 kubenswrapper[2563]: E0910 01:04:16.077437    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t_llm(d66ca905-636b-4e4e-94ce-e26913e3f8a6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t" podUID="d66ca905-636b-4e4e-94ce-e26913e3f8a6"
Sep 10 01:04:25.110913 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:25.110875    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t_d66ca905-636b-4e4e-94ce-e26913e3f8a6/main/5.log"
Sep 10 01:04:25.229285 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:25.229258    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t_d66ca905-636b-4e4e-94ce-e26913e3f8a6/main/5.log"
Sep 10 01:04:29.076695 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:29.076660    2563 scope.go:117] "RemoveContainer" containerID="b090d8d22885556b1950653a061fee464158d77029123f4387435a1bac0a87ed"
Sep 10 01:04:29.077108 ip-10-0-138-204 kubenswrapper[2563]: E0910 01:04:29.076846    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t_llm(d66ca905-636b-4e4e-94ce-e26913e3f8a6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t" podUID="d66ca905-636b-4e4e-94ce-e26913e3f8a6"
Sep 10 01:04:31.698156 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:31.698122    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-m6mg5_3d3520e0-e82d-4178-9a58-07a44abccd49/manager/0.log"
Sep 10 01:04:31.818601 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:31.818570    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-f7z4t_ed4f6ebe-d3d9-4014-8648-9536f3734cde/manager/0.log"
Sep 10 01:04:31.941228 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:31.941195    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-87f8d868d-c77l7_6c5f6fed-3b98-4569-8e26-e4918ff734a2/manager/0.log"
Sep 10 01:04:32.061110 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:32.061035    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-6nlmk_deb73139-28d2-4ed7-bd29-d97a262c17a6/server/0.log"
Sep 10 01:04:32.179210 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:32.179178    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-njqxx_b0d433ff-4b9c-41cc-a06b-12fb4c96c6d7/manager/0.log"
Sep 10 01:04:32.428480 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:32.428451    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5dc88f9465-bmqsl_8c11b938-a977-4632-8f25-99f8645a495e/copy-manifests/0.log"
Sep 10 01:04:32.434584 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:32.434556    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5dc88f9465-bmqsl_8c11b938-a977-4632-8f25-99f8645a495e/manager/0.log"
Sep 10 01:04:32.906256 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:32.906225    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6dc78795f5-h2rxm_2dc98840-2155-4e6c-b015-af85591fe40d/maas-api/0.log"
Sep 10 01:04:34.030745 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:34.030711    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-m979s_acfc4eb4-0449-468b-88cf-acfe8976858b/manager/0.log"
Sep 10 01:04:34.140144 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:34.140111    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-5vh2f_1c8597bd-5ba5-4fc5-ab46-f7d7d8bedd0f/manager/0.log"
Sep 10 01:04:34.604541 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:34.604509    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-rzw4v_f9f4d856-cc01-4df4-bf3f-f103bea1d309/manager/0.log"
Sep 10 01:04:35.630940 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:35.630903    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-xzbz7_dcac140e-784c-4cd5-ba9d-b7742fedc271/payload-processing/0.log"
Sep 10 01:04:36.303667 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:36.303597    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-nkwlr_cd2a4e9a-f207-4b1d-998a-591801f95098/main/0.log"
Sep 10 01:04:36.637253 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:36.637228    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t_d66ca905-636b-4e4e-94ce-e26913e3f8a6/main/5.log"
Sep 10 01:04:43.076714 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:43.076679    2563 scope.go:117] "RemoveContainer" containerID="b090d8d22885556b1950653a061fee464158d77029123f4387435a1bac0a87ed"
Sep 10 01:04:43.077231 ip-10-0-138-204 kubenswrapper[2563]: E0910 01:04:43.076871    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t_llm(d66ca905-636b-4e4e-94ce-e26913e3f8a6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t" podUID="d66ca905-636b-4e4e-94ce-e26913e3f8a6"
Sep 10 01:04:44.010159 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:44.010127    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nnwxp_1cee7771-6381-4fad-8c1e-ad1098fe409b/console-operator/1.log"
Sep 10 01:04:44.011144 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:44.011119    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nnwxp_1cee7771-6381-4fad-8c1e-ad1098fe409b/console-operator/1.log"
Sep 10 01:04:45.524842 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:45.524810    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-l4w6p_00cec203-bdcf-4e75-acfa-29e1b784840e/global-pull-secret-syncer/0.log"
Sep 10 01:04:45.634745 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:45.634710    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qhbzc_caff9962-15c8-4c05-ad49-a748c94a3b85/konnectivity-agent/0.log"
Sep 10 01:04:45.733357 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:45.733316    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-204.ec2.internal_4ce798d5035df86fd4210aa6d5fc499e/haproxy/0.log"
Sep 10 01:04:50.642635 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:50.642600    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-m979s_acfc4eb4-0449-468b-88cf-acfe8976858b/manager/0.log"
Sep 10 01:04:50.670106 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:50.670075    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-5vh2f_1c8597bd-5ba5-4fc5-ab46-f7d7d8bedd0f/manager/0.log"
Sep 10 01:04:50.794332 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:50.794294    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-rzw4v_f9f4d856-cc01-4df4-bf3f-f103bea1d309/manager/0.log"
Sep 10 01:04:52.498473 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:52.498395    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-4qhvr_784a9928-9ed1-4f75-afdb-c42ee2030777/cluster-monitoring-operator/0.log"
Sep 10 01:04:52.522671 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:52.522646    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-2d77r_5b956a55-24be-4dd1-b187-4fa5c389dc14/kube-state-metrics/0.log"
Sep 10 01:04:52.543229 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:52.543191    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-2d77r_5b956a55-24be-4dd1-b187-4fa5c389dc14/kube-rbac-proxy-main/0.log"
Sep 10 01:04:52.563543 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:52.563518    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-2d77r_5b956a55-24be-4dd1-b187-4fa5c389dc14/kube-rbac-proxy-self/0.log"
Sep 10 01:04:52.664424 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:52.664395    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2zflc_70e3b137-1fc5-47a2-b0d1-86af94a809e5/node-exporter/0.log"
Sep 10 01:04:52.685951 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:52.685926    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2zflc_70e3b137-1fc5-47a2-b0d1-86af94a809e5/kube-rbac-proxy/0.log"
Sep 10 01:04:52.709993 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:52.709956    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2zflc_70e3b137-1fc5-47a2-b0d1-86af94a809e5/init-textfile/0.log"
Sep 10 01:04:52.920175 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:52.920142    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-lxhqc_6ca637d2-53c4-424d-8f3b-c6ca6ea03d2c/kube-rbac-proxy-main/0.log"
Sep 10 01:04:52.943128 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:52.943094    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-lxhqc_6ca637d2-53c4-424d-8f3b-c6ca6ea03d2c/kube-rbac-proxy-self/0.log"
Sep 10 01:04:52.969136 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:52.969089    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-lxhqc_6ca637d2-53c4-424d-8f3b-c6ca6ea03d2c/openshift-state-metrics/0.log"
Sep 10 01:04:53.006599 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:53.006572    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b629bec7-e856-4a92-afe8-df3dba62dbb9/prometheus/0.log"
Sep 10 01:04:53.027149 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:53.027118    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b629bec7-e856-4a92-afe8-df3dba62dbb9/config-reloader/0.log"
Sep 10 01:04:53.052543 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:53.052458    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b629bec7-e856-4a92-afe8-df3dba62dbb9/thanos-sidecar/0.log"
Sep 10 01:04:53.075900 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:53.075861    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b629bec7-e856-4a92-afe8-df3dba62dbb9/kube-rbac-proxy-web/0.log"
Sep 10 01:04:53.098641 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:53.098615    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b629bec7-e856-4a92-afe8-df3dba62dbb9/kube-rbac-proxy/0.log"
Sep 10 01:04:53.120687 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:53.120659    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b629bec7-e856-4a92-afe8-df3dba62dbb9/kube-rbac-proxy-thanos/0.log"
Sep 10 01:04:53.141810 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:53.141778    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b629bec7-e856-4a92-afe8-df3dba62dbb9/init-config-reloader/0.log"
Sep 10 01:04:53.170057 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:53.170009    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-7ckhj_b9d98f37-c819-4628-b504-00d4ce21a7ab/prometheus-operator/0.log"
Sep 10 01:04:53.198733 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:53.198704    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-7ckhj_b9d98f37-c819-4628-b504-00d4ce21a7ab/kube-rbac-proxy/0.log"
Sep 10 01:04:53.228087 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:53.228060    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-kp4ws_6f40c51f-1831-47b7-8b0a-e8dcf78bfed0/prometheus-operator-admission-webhook/0.log"
Sep 10 01:04:54.730804 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:54.730776    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-gzrtx_b71eb054-e5f2-4f16-b9a3-1253c28778ca/networking-console-plugin/0.log"
Sep 10 01:04:55.076597 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:55.076515    2563 scope.go:117] "RemoveContainer" containerID="b090d8d22885556b1950653a061fee464158d77029123f4387435a1bac0a87ed"
Sep 10 01:04:55.076748 ip-10-0-138-204 kubenswrapper[2563]: E0910 01:04:55.076701    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t_llm(d66ca905-636b-4e4e-94ce-e26913e3f8a6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t" podUID="d66ca905-636b-4e4e-94ce-e26913e3f8a6"
Sep 10 01:04:55.347408 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:55.347380    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nnwxp_1cee7771-6381-4fad-8c1e-ad1098fe409b/console-operator/1.log"
Sep 10 01:04:55.353573 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:55.353548    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nnwxp_1cee7771-6381-4fad-8c1e-ad1098fe409b/console-operator/2.log"
Sep 10 01:04:55.909797 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:55.909771    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5fd968dd46-n9nq6_6613e142-5fea-4fc4-a420-1efccaf5d344/console/0.log"
Sep 10 01:04:55.989966 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:55.989927    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-sb69v/perf-node-gather-daemonset-56sxn"]
Sep 10 01:04:55.993645 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:55.993625    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sb69v/perf-node-gather-daemonset-56sxn"
Sep 10 01:04:55.996068 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:55.996042    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sb69v\"/\"kube-root-ca.crt\""
Sep 10 01:04:55.996068 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:55.996058    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-sb69v\"/\"default-dockercfg-9hx98\""
Sep 10 01:04:55.996240 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:55.996043    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sb69v\"/\"openshift-service-ca.crt\""
Sep 10 01:04:56.002585 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:56.002562    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sb69v/perf-node-gather-daemonset-56sxn"]
Sep 10 01:04:56.123266 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:56.123221    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/61eb05c1-07b0-4f4f-9a12-9ea402dfb365-lib-modules\") pod \"perf-node-gather-daemonset-56sxn\" (UID: \"61eb05c1-07b0-4f4f-9a12-9ea402dfb365\") " pod="openshift-must-gather-sb69v/perf-node-gather-daemonset-56sxn"
Sep 10 01:04:56.123266 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:56.123271    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkkdw\" (UniqueName: \"kubernetes.io/projected/61eb05c1-07b0-4f4f-9a12-9ea402dfb365-kube-api-access-nkkdw\") pod \"perf-node-gather-daemonset-56sxn\" (UID: \"61eb05c1-07b0-4f4f-9a12-9ea402dfb365\") " pod="openshift-must-gather-sb69v/perf-node-gather-daemonset-56sxn"
Sep 10 01:04:56.124051 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:56.124013    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/61eb05c1-07b0-4f4f-9a12-9ea402dfb365-proc\") pod \"perf-node-gather-daemonset-56sxn\" (UID: \"61eb05c1-07b0-4f4f-9a12-9ea402dfb365\") " pod="openshift-must-gather-sb69v/perf-node-gather-daemonset-56sxn"
Sep 10 01:04:56.124196 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:56.124090    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61eb05c1-07b0-4f4f-9a12-9ea402dfb365-sys\") pod \"perf-node-gather-daemonset-56sxn\" (UID: \"61eb05c1-07b0-4f4f-9a12-9ea402dfb365\") " pod="openshift-must-gather-sb69v/perf-node-gather-daemonset-56sxn"
Sep 10 01:04:56.124196 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:56.124139    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/61eb05c1-07b0-4f4f-9a12-9ea402dfb365-podres\") pod \"perf-node-gather-daemonset-56sxn\" (UID: \"61eb05c1-07b0-4f4f-9a12-9ea402dfb365\") " pod="openshift-must-gather-sb69v/perf-node-gather-daemonset-56sxn"
Sep 10 01:04:56.225236 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:56.225148    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/61eb05c1-07b0-4f4f-9a12-9ea402dfb365-lib-modules\") pod \"perf-node-gather-daemonset-56sxn\" (UID: \"61eb05c1-07b0-4f4f-9a12-9ea402dfb365\") " pod="openshift-must-gather-sb69v/perf-node-gather-daemonset-56sxn"
Sep 10 01:04:56.225236 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:56.225187    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkkdw\" (UniqueName: \"kubernetes.io/projected/61eb05c1-07b0-4f4f-9a12-9ea402dfb365-kube-api-access-nkkdw\") pod \"perf-node-gather-daemonset-56sxn\" (UID: \"61eb05c1-07b0-4f4f-9a12-9ea402dfb365\") " pod="openshift-must-gather-sb69v/perf-node-gather-daemonset-56sxn"
Sep 10 01:04:56.225426 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:56.225242    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/61eb05c1-07b0-4f4f-9a12-9ea402dfb365-proc\") pod \"perf-node-gather-daemonset-56sxn\" (UID: \"61eb05c1-07b0-4f4f-9a12-9ea402dfb365\") " pod="openshift-must-gather-sb69v/perf-node-gather-daemonset-56sxn"
Sep 10 01:04:56.225426 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:56.225264    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61eb05c1-07b0-4f4f-9a12-9ea402dfb365-sys\") pod \"perf-node-gather-daemonset-56sxn\" (UID: \"61eb05c1-07b0-4f4f-9a12-9ea402dfb365\") " pod="openshift-must-gather-sb69v/perf-node-gather-daemonset-56sxn"
Sep 10 01:04:56.225426 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:56.225285    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/61eb05c1-07b0-4f4f-9a12-9ea402dfb365-podres\") pod \"perf-node-gather-daemonset-56sxn\" (UID: \"61eb05c1-07b0-4f4f-9a12-9ea402dfb365\") " pod="openshift-must-gather-sb69v/perf-node-gather-daemonset-56sxn"
Sep 10 01:04:56.225426 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:56.225352    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/61eb05c1-07b0-4f4f-9a12-9ea402dfb365-proc\") pod \"perf-node-gather-daemonset-56sxn\" (UID: \"61eb05c1-07b0-4f4f-9a12-9ea402dfb365\") " pod="openshift-must-gather-sb69v/perf-node-gather-daemonset-56sxn"
Sep 10 01:04:56.225426 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:56.225355    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61eb05c1-07b0-4f4f-9a12-9ea402dfb365-sys\") pod \"perf-node-gather-daemonset-56sxn\" (UID: \"61eb05c1-07b0-4f4f-9a12-9ea402dfb365\") " pod="openshift-must-gather-sb69v/perf-node-gather-daemonset-56sxn"
Sep 10 01:04:56.225426 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:56.225352    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/61eb05c1-07b0-4f4f-9a12-9ea402dfb365-lib-modules\") pod \"perf-node-gather-daemonset-56sxn\" (UID: \"61eb05c1-07b0-4f4f-9a12-9ea402dfb365\") " pod="openshift-must-gather-sb69v/perf-node-gather-daemonset-56sxn"
Sep 10 01:04:56.225426 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:56.225411    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/61eb05c1-07b0-4f4f-9a12-9ea402dfb365-podres\") pod \"perf-node-gather-daemonset-56sxn\" (UID: \"61eb05c1-07b0-4f4f-9a12-9ea402dfb365\") " pod="openshift-must-gather-sb69v/perf-node-gather-daemonset-56sxn"
Sep 10 01:04:56.232428 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:56.232404    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkkdw\" (UniqueName: \"kubernetes.io/projected/61eb05c1-07b0-4f4f-9a12-9ea402dfb365-kube-api-access-nkkdw\") pod \"perf-node-gather-daemonset-56sxn\" (UID: \"61eb05c1-07b0-4f4f-9a12-9ea402dfb365\") " pod="openshift-must-gather-sb69v/perf-node-gather-daemonset-56sxn"
Sep 10 01:04:56.304445 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:56.304402    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sb69v/perf-node-gather-daemonset-56sxn"
Sep 10 01:04:56.575228 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:56.575200    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sb69v/perf-node-gather-daemonset-56sxn"]
Sep 10 01:04:56.576909 ip-10-0-138-204 kubenswrapper[2563]: W0910 01:04:56.576880    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod61eb05c1_07b0_4f4f_9a12_9ea402dfb365.slice/crio-2d155b94842560bd4509f3b1f6406a2f2408e9a74174c7cdbf14e5bac7c24c4e WatchSource:0}: Error finding container 2d155b94842560bd4509f3b1f6406a2f2408e9a74174c7cdbf14e5bac7c24c4e: Status 404 returned error can't find the container with id 2d155b94842560bd4509f3b1f6406a2f2408e9a74174c7cdbf14e5bac7c24c4e
Sep 10 01:04:57.519737 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:57.519702    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sb69v/perf-node-gather-daemonset-56sxn" event={"ID":"61eb05c1-07b0-4f4f-9a12-9ea402dfb365","Type":"ContainerStarted","Data":"8d8802e8b9193ad0c59171a1f9877207f5c46b901fdb8b767a06dbed8bc04756"}
Sep 10 01:04:57.520199 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:57.519755    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sb69v/perf-node-gather-daemonset-56sxn" event={"ID":"61eb05c1-07b0-4f4f-9a12-9ea402dfb365","Type":"ContainerStarted","Data":"2d155b94842560bd4509f3b1f6406a2f2408e9a74174c7cdbf14e5bac7c24c4e"}
Sep 10 01:04:57.520199 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:57.519837    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-sb69v/perf-node-gather-daemonset-56sxn"
Sep 10 01:04:57.732542 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:57.732513    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kw2lh_58138b99-90e8-4b0a-89bf-891604fbaca8/dns/0.log"
Sep 10 01:04:57.754626 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:57.754595    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kw2lh_58138b99-90e8-4b0a-89bf-891604fbaca8/kube-rbac-proxy/0.log"
Sep 10 01:04:57.920249 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:57.920222    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-p5zpt_27351a7f-151c-4d9a-bddd-4ebaa3a42ade/dns-node-resolver/0.log"
Sep 10 01:04:58.485814 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:58.485782    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tr2kj_8686a32b-9d47-4866-aae0-89f0b6575ff3/node-ca/0.log"
Sep 10 01:04:59.541810 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:04:59.541783    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-xzbz7_dcac140e-784c-4cd5-ba9d-b7742fedc271/payload-processing/0.log"
Sep 10 01:05:00.232589 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:05:00.232560    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sgpwp_96504ff2-84b9-442b-a77e-254cbe9d8e5a/serve-healthcheck-canary/0.log"
Sep 10 01:05:00.698118 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:05:00.698084    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-kdm6r_501b4642-4933-4505-afe4-4446b75ce2ee/insights-operator/1.log"
Sep 10 01:05:00.698522 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:05:00.698157    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-kdm6r_501b4642-4933-4505-afe4-4446b75ce2ee/insights-operator/0.log"
Sep 10 01:05:00.718579 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:05:00.718549    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-k9rtv_d794c501-ec2b-419a-aefa-b61f7dedad8e/kube-rbac-proxy/0.log"
Sep 10 01:05:00.738797 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:05:00.738766    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-k9rtv_d794c501-ec2b-419a-aefa-b61f7dedad8e/exporter/0.log"
Sep 10 01:05:00.759249 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:05:00.759221    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-k9rtv_d794c501-ec2b-419a-aefa-b61f7dedad8e/extractor/0.log"
Sep 10 01:05:02.960590 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:05:02.960563    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-m6mg5_3d3520e0-e82d-4178-9a58-07a44abccd49/manager/0.log"
Sep 10 01:05:03.001959 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:05:03.001897    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-f7z4t_ed4f6ebe-d3d9-4014-8648-9536f3734cde/manager/0.log"
Sep 10 01:05:03.037738 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:05:03.037709    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-87f8d868d-c77l7_6c5f6fed-3b98-4569-8e26-e4918ff734a2/manager/0.log"
Sep 10 01:05:03.056820 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:05:03.056794    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-6nlmk_deb73139-28d2-4ed7-bd29-d97a262c17a6/server/0.log"
Sep 10 01:05:03.102052 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:05:03.102019    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-njqxx_b0d433ff-4b9c-41cc-a06b-12fb4c96c6d7/manager/0.log"
Sep 10 01:05:03.186965 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:05:03.186937    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5dc88f9465-bmqsl_8c11b938-a977-4632-8f25-99f8645a495e/manager/0.log"
Sep 10 01:05:03.207356 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:05:03.207324    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5dc88f9465-bmqsl_8c11b938-a977-4632-8f25-99f8645a495e/copy-manifests/0.log"
Sep 10 01:05:03.536778 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:05:03.536745    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-sb69v/perf-node-gather-daemonset-56sxn"
Sep 10 01:05:03.552831 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:05:03.552771    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-sb69v/perf-node-gather-daemonset-56sxn" podStartSLOduration=8.552755215 podStartE2EDuration="8.552755215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 01:04:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 01:04:57.534264004 +0000 UTC m=+914.031563565" watchObservedRunningTime="2026-09-10 01:05:03.552755215 +0000 UTC m=+920.050054842"
Sep 10 01:05:04.369020 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:05:04.368977    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-56bf56cd54-w6grw_3a8425cd-4ccc-4117-8d29-b64f424da3be/manager/0.log"
Sep 10 01:05:06.188449 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:05:06.188417    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-xzbz7_dcac140e-784c-4cd5-ba9d-b7742fedc271/payload-processing/0.log"
Sep 10 01:05:07.585737 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:05:07.585650    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-xzbz7_dcac140e-784c-4cd5-ba9d-b7742fedc271/payload-processing/0.log"
Sep 10 01:05:08.076768 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:05:08.076736    2563 scope.go:117] "RemoveContainer" containerID="b090d8d22885556b1950653a061fee464158d77029123f4387435a1bac0a87ed"
Sep 10 01:05:08.076968 ip-10-0-138-204 kubenswrapper[2563]: E0910 01:05:08.076942    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t_llm(d66ca905-636b-4e4e-94ce-e26913e3f8a6)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-l4d4t" podUID="d66ca905-636b-4e4e-94ce-e26913e3f8a6"
Sep 10 01:05:10.961701 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:05:10.961662    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-9zt49_7a312259-8fb8-4bb4-9304-0b5aac8240f2/kube-multus/0.log"
Sep 10 01:05:10.985060 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:05:10.985025    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6nzvr_9f66caff-9501-4b7e-8986-9161a3ae70e3/kube-multus-additional-cni-plugins/0.log"
Sep 10 01:05:11.005924 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:05:11.005895    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6nzvr_9f66caff-9501-4b7e-8986-9161a3ae70e3/egress-router-binary-copy/0.log"
Sep 10 01:05:11.026552 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:05:11.026522    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6nzvr_9f66caff-9501-4b7e-8986-9161a3ae70e3/cni-plugins/0.log"
Sep 10 01:05:11.048344 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:05:11.048318    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6nzvr_9f66caff-9501-4b7e-8986-9161a3ae70e3/bond-cni-plugin/0.log"
Sep 10 01:05:11.068758 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:05:11.068735    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6nzvr_9f66caff-9501-4b7e-8986-9161a3ae70e3/routeoverride-cni/0.log"
Sep 10 01:05:11.091721 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:05:11.091698    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6nzvr_9f66caff-9501-4b7e-8986-9161a3ae70e3/whereabouts-cni-bincopy/0.log"
Sep 10 01:05:11.113692 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:05:11.113664    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6nzvr_9f66caff-9501-4b7e-8986-9161a3ae70e3/whereabouts-cni/0.log"
Sep 10 01:05:11.532547 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:05:11.532424    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-8ltcj_fc958bde-e5aa-468a-ae10-0c49f9438168/network-metrics-daemon/0.log"
Sep 10 01:05:11.552604 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:05:11.552579    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-8ltcj_fc958bde-e5aa-468a-ae10-0c49f9438168/kube-rbac-proxy/0.log"
Sep 10 01:05:12.433421 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:05:12.433393    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8nwz8_981f2019-8639-47c8-b1d3-f2dbb02acd15/ovn-controller/0.log"
Sep 10 01:05:12.452345 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:05:12.452314    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8nwz8_981f2019-8639-47c8-b1d3-f2dbb02acd15/ovn-acl-logging/0.log"
Sep 10 01:05:12.475266 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:05:12.475236    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8nwz8_981f2019-8639-47c8-b1d3-f2dbb02acd15/kube-rbac-proxy-node/0.log"
Sep 10 01:05:12.496525 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:05:12.496497    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8nwz8_981f2019-8639-47c8-b1d3-f2dbb02acd15/kube-rbac-proxy-ovn-metrics/0.log"
Sep 10 01:05:12.516561 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:05:12.516531    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8nwz8_981f2019-8639-47c8-b1d3-f2dbb02acd15/northd/0.log"
Sep 10 01:05:12.540038 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:05:12.540008    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8nwz8_981f2019-8639-47c8-b1d3-f2dbb02acd15/nbdb/0.log"
Sep 10 01:05:12.562016 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:05:12.561970    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8nwz8_981f2019-8639-47c8-b1d3-f2dbb02acd15/sbdb/0.log"
Sep 10 01:05:12.603144 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:05:12.603112    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8nwz8_981f2019-8639-47c8-b1d3-f2dbb02acd15/ovnkube-controller/0.log"
Sep 10 01:05:14.074451 ip-10-0-138-204 kubenswrapper[2563]: I0910 01:05:14.074416    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-cvjrx_50d35dde-26ab-4a1a-8f6a-2d663df04bc5/network-check-target-container/0.log"